Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260809 14:29:37.363487 31729 test_util.cc:268] Using random seed: 289022356
2026-08-09T14:29:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T14:29:37Z Disabled control of system clock
I20260809 14:29:37.418833 31729 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
/tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.252.126:43223
--webserver_interface=127.30.252.126
--webserver_port=0
--builtin_ntp_servers=127.30.252.84:46315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260809 14:29:37.715772 31738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 14:29:37.716382 31738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 14:29:37.716907 31738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 14:29:37.747189 31738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 14:29:37.747457 31738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 14:29:37.747764 31738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 14:29:37.748023 31738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 14:29:37.781677 31738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.252.84:46315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.252.126:43223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.252.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 13:36:37 UTC on ccycloud.quasar-iqymvz.root.comops.site
build id 4576526
TSAN enabled
I20260809 14:29:37.783013 31738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 14:29:37.784623 31738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 14:29:37.802127 31748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 14:29:37.802268 31745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:37.803436 31738 server_base.cc:1034] running on GCE node
W20260809 14:29:37.803385 31744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:39.003698 31738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 14:29:39.007055 31738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 14:29:39.008565 31738 hybrid_clock.cc:648] HybridClock initialized: now 1786285779008504 us; error 78 us; skew 500 ppm
I20260809 14:29:39.009377 31738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 14:29:39.015867 31738 webserver.cc:460] Webserver started at http://127.30.252.126:39845/ using document root <none> and password file <none>
I20260809 14:29:39.016893 31738 fs_manager.cc:359] Metadata directory not provided
I20260809 14:29:39.017091 31738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 14:29:39.017532 31738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 14:29:39.022393 31738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/instance:
uuid: "9d10512ac8b3406098fad1f7da012ab3"
format_stamp: "Formatted at 2026-08-09 14:29:39 on dist-test-slave-10pc"
I20260809 14:29:39.023455 31738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal/instance:
uuid: "9d10512ac8b3406098fad1f7da012ab3"
format_stamp: "Formatted at 2026-08-09 14:29:39 on dist-test-slave-10pc"
I20260809 14:29:39.030777 31738 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260809 14:29:39.036270 31756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:39.037295 31738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 14:29:39.037638 31738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data,/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
uuid: "9d10512ac8b3406098fad1f7da012ab3"
format_stamp: "Formatted at 2026-08-09 14:29:39 on dist-test-slave-10pc"
I20260809 14:29:39.038026 31738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 14:29:39.096366 31738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 14:29:39.098016 31738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 14:29:39.098495 31738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 14:29:39.169909 31738 rpc_server.cc:307] RPC server started. Bound to: 127.30.252.126:43223
I20260809 14:29:39.170010 31812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.252.126:43223 every 8 connection(s)
I20260809 14:29:39.172670 31738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/info.pb
I20260809 14:29:39.178063 31813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:39.181905 31729 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu as pid 31738
I20260809 14:29:39.182425 31729 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal/instance
I20260809 14:29:39.188051 31729 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
/tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.252.125:42169
--webserver_interface=127.30.252.125
--webserver_port=0
--builtin_ntp_servers=127.30.252.84:46315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 14:29:39.194399 31813 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } has no permanent_uuid. Determining permanent_uuid...
I20260809 14:29:39.222788 31813 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
W20260809 14:29:39.225229 31760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.252.125:42169: connect: Connection refused (error 111)
W20260809 14:29:39.229144 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.125:42169: Network error: Client connection negotiation failed: client connection to 127.30.252.125:42169: connect: Connection refused (error 111)
I20260809 14:29:39.278626 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } attempt: 1
W20260809 14:29:39.282366 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.125:42169: Network error: Client connection negotiation failed: client connection to 127.30.252.125:42169: connect: Connection refused (error 111)
I20260809 14:29:39.350911 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } attempt: 2
W20260809 14:29:39.354765 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.125:42169: Network error: Client connection negotiation failed: client connection to 127.30.252.125:42169: connect: Connection refused (error 111)
I20260809 14:29:39.446271 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } attempt: 3
W20260809 14:29:39.450030 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.125:42169: Network error: Client connection negotiation failed: client connection to 127.30.252.125:42169: connect: Connection refused (error 111)
W20260809 14:29:39.508138 31816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 14:29:39.508718 31816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 14:29:39.509124 31816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 14:29:39.540323 31816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 14:29:39.540632 31816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 14:29:39.540829 31816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 14:29:39.541082 31816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 14:29:39.575523 31816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.252.84:46315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.252.125:42169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.252.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 13:36:37 UTC on ccycloud.quasar-iqymvz.root.comops.site
build id 4576526
TSAN enabled
I20260809 14:29:39.576720 31816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 14:29:39.578235 31816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 14:29:39.593485 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } attempt: 4
W20260809 14:29:39.597677 31827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 14:29:39.598237 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.125:42169: Network error: Client connection negotiation failed: client connection to 127.30.252.125:42169: connect: Connection refused (error 111)
I20260809 14:29:39.897739 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } attempt: 5
W20260809 14:29:39.903039 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.125:42169: Network error: Client connection negotiation failed: client connection to 127.30.252.125:42169: connect: Connection refused (error 111)
I20260809 14:29:40.450492 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } attempt: 6
W20260809 14:29:40.458633 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.125:42169: Network error: Client connection negotiation failed: client connection to 127.30.252.125:42169: connect: Connection refused (error 111)
W20260809 14:29:39.598771 31826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 14:29:39.599515 31829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:39.599699 31816 server_base.cc:1034] running on GCE node
I20260809 14:29:40.798975 31816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 14:29:40.802134 31816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 14:29:40.803560 31816 hybrid_clock.cc:648] HybridClock initialized: now 1786285780803500 us; error 82 us; skew 500 ppm
I20260809 14:29:40.804348 31816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 14:29:40.815613 31816 webserver.cc:460] Webserver started at http://127.30.252.125:33167/ using document root <none> and password file <none>
I20260809 14:29:40.816561 31816 fs_manager.cc:359] Metadata directory not provided
I20260809 14:29:40.816751 31816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 14:29:40.817184 31816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 14:29:40.822013 31816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/data/instance:
uuid: "852dbd0962ae4211923851c7a632768e"
format_stamp: "Formatted at 2026-08-09 14:29:40 on dist-test-slave-10pc"
I20260809 14:29:40.823067 31816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/wal/instance:
uuid: "852dbd0962ae4211923851c7a632768e"
format_stamp: "Formatted at 2026-08-09 14:29:40 on dist-test-slave-10pc"
I20260809 14:29:40.830269 31816 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260809 14:29:40.835891 31839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:40.836984 31816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 14:29:40.837330 31816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/data,/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/wal
uuid: "852dbd0962ae4211923851c7a632768e"
format_stamp: "Formatted at 2026-08-09 14:29:40 on dist-test-slave-10pc"
I20260809 14:29:40.837707 31816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 14:29:40.887624 31816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 14:29:40.889161 31816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 14:29:40.889657 31816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 14:29:40.961484 31816 rpc_server.cc:307] RPC server started. Bound to: 127.30.252.125:42169
I20260809 14:29:40.961558 31899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.252.125:42169 every 8 connection(s)
I20260809 14:29:40.964193 31816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/data/info.pb
I20260809 14:29:40.969295 31900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:40.972162 31729 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu as pid 31816
I20260809 14:29:40.972621 31729 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-1/wal/instance
I20260809 14:29:40.974782 31729 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
/tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.252.124:34903
--webserver_interface=127.30.252.124
--webserver_port=0
--builtin_ntp_servers=127.30.252.84:46315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 14:29:40.980324 31900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } has no permanent_uuid. Determining permanent_uuid...
I20260809 14:29:41.005594 31900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
I20260809 14:29:41.022244 31900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
W20260809 14:29:41.024577 31842 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
W20260809 14:29:41.028860 31900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.078395 31900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 1
W20260809 14:29:41.082093 31900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.150574 31900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 2
W20260809 14:29:41.154383 31900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.245857 31900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 3
W20260809 14:29:41.249663 31900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
W20260809 14:29:41.288337 31902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 14:29:41.289017 31902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 14:29:41.289500 31902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 14:29:41.320086 31902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 14:29:41.320437 31902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 14:29:41.320735 31902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 14:29:41.321035 31902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 14:29:41.356554 31902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.252.84:46315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.252.124:34903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.252.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 13:36:37 UTC on ccycloud.quasar-iqymvz.root.comops.site
build id 4576526
TSAN enabled
I20260809 14:29:41.357961 31902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 14:29:41.359745 31902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 14:29:41.376780 31913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:41.393107 31900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 4
W20260809 14:29:41.397465 31900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.519053 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } attempt: 7
I20260809 14:29:41.532007 31813 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
W20260809 14:29:41.538193 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.596542 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 1
W20260809 14:29:41.601002 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.682484 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 2
W20260809 14:29:41.686895 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.696830 31900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 5
W20260809 14:29:41.701082 31900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.772269 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 3
W20260809 14:29:41.776582 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:41.916993 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 4
W20260809 14:29:41.921214 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:42.204583 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 5
W20260809 14:29:42.208245 31813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
I20260809 14:29:42.248492 31900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 6
W20260809 14:29:42.257575 31900 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.252.124:34903: Network error: Client connection negotiation failed: client connection to 127.30.252.124:34903: connect: Connection refused (error 111)
W20260809 14:29:41.377046 31916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:41.379354 31902 server_base.cc:1034] running on GCE node
W20260809 14:29:41.377763 31914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:42.577163 31902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 14:29:42.579718 31902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 14:29:42.581106 31902 hybrid_clock.cc:648] HybridClock initialized: now 1786285782581067 us; error 56 us; skew 500 ppm
I20260809 14:29:42.581943 31902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 14:29:42.588425 31902 webserver.cc:460] Webserver started at http://127.30.252.124:36063/ using document root <none> and password file <none>
I20260809 14:29:42.589414 31902 fs_manager.cc:359] Metadata directory not provided
I20260809 14:29:42.589607 31902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 14:29:42.590046 31902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 14:29:42.595176 31902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/data/instance:
uuid: "ee93a411a1b04d38998018f8c6be59e0"
format_stamp: "Formatted at 2026-08-09 14:29:42 on dist-test-slave-10pc"
I20260809 14:29:42.596246 31902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/wal/instance:
uuid: "ee93a411a1b04d38998018f8c6be59e0"
format_stamp: "Formatted at 2026-08-09 14:29:42 on dist-test-slave-10pc"
I20260809 14:29:42.603571 31902 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260809 14:29:42.609551 31929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:42.610683 31902 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 14:29:42.610971 31902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/data,/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/wal
uuid: "ee93a411a1b04d38998018f8c6be59e0"
format_stamp: "Formatted at 2026-08-09 14:29:42 on dist-test-slave-10pc"
I20260809 14:29:42.611279 31902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 14:29:42.655076 31902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 14:29:42.656635 31902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 14:29:42.657050 31902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 14:29:42.730602 31902 rpc_server.cc:307] RPC server started. Bound to: 127.30.252.124:34903
I20260809 14:29:42.730712 31984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.252.124:34903 every 8 connection(s)
I20260809 14:29:42.733517 31902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/data/info.pb
I20260809 14:29:42.738691 31985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:42.741376 31729 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu as pid 31902
I20260809 14:29:42.741775 31729 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-2/wal/instance
I20260809 14:29:42.752857 31985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } has no permanent_uuid. Determining permanent_uuid...
I20260809 14:29:42.761741 31813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 6
I20260809 14:29:42.797224 31985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
I20260809 14:29:42.809876 31813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Bootstrap starting.
I20260809 14:29:42.816854 31985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
I20260809 14:29:42.819370 31813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:42.821913 31813 log.cc:824] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Log is configured to *not* fsync() on all Append() calls
I20260809 14:29:42.828589 31813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: No bootstrap required, opened a new log
I20260809 14:29:42.840564 31985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Bootstrap starting.
I20260809 14:29:42.848889 31985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:42.850844 31985 log.cc:824] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Log is configured to *not* fsync() on all Append() calls
I20260809 14:29:42.856279 31985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: No bootstrap required, opened a new log
I20260809 14:29:42.858886 31813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:42.859594 31813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:42.859910 31813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d10512ac8b3406098fad1f7da012ab3, State: Initialized, Role: FOLLOWER
I20260809 14:29:42.860862 31813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:42.864142 32001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:42.865299 32001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:42.865487 31813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 14:29:42.884500 31985 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:42.885462 31985 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:42.885874 31985 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee93a411a1b04d38998018f8c6be59e0, State: Initialized, Role: FOLLOWER
I20260809 14:29:42.887231 31985 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:42.891230 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:42.892290 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:42.893226 31985 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 14:29:42.906810 32014 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 14:29:42.907302 32014 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 14:29:42.912479 31935 tablet.cc:2369] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 14:29:42.902070 31738 master_runner.cc:186] Error getting master registration for 127.30.252.125:42169: OK, instance_id { permanent_uuid: "852dbd0962ae4211923851c7a632768e" instance_seqno: 1786285780941413 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 14:29:42.937290 31738 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 14:29:42.940580 32031 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 14:29:42.940904 32031 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 14:29:42.944316 31902 master_runner.cc:186] Error getting master registration for 127.30.252.125:42169: OK, instance_id { permanent_uuid: "852dbd0962ae4211923851c7a632768e" instance_seqno: 1786285780941413 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 14:29:42.989466 31902 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 14:29:43.107829 31762 tablet.cc:2369] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 14:29:43.113581 32001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:43.114193 32001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:43.118384 32001 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:43.118615 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:43.119658 31959 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d10512ac8b3406098fad1f7da012ab3 in term 0.
I20260809 14:29:43.121035 31759 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0; no voters:
I20260809 14:29:43.122098 32001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:43.122809 32001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:43.123478 32001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:43.137848 32001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:43.143594 32001 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 1 election: Requested vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:43.144740 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0"
I20260809 14:29:43.145408 31959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:43.153091 31959 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d10512ac8b3406098fad1f7da012ab3 in term 1.
I20260809 14:29:43.154181 31759 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0; no voters:
I20260809 14:29:43.155426 32001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:43.160302 32001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 9d10512ac8b3406098fad1f7da012ab3, State: Running, Role: LEADER
I20260809 14:29:43.162671 32001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:43.188118 32033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d10512ac8b3406098fad1f7da012ab3. Latest consensus state: current_term: 1 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:43.189232 32033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:43.205996 32035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 14:29:43.226550 32035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 14:29:43.272356 31959 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Refusing update from remote peer 9d10512ac8b3406098fad1f7da012ab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 14:29:43.274506 32033 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:29:43.312399 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d10512ac8b3406098fad1f7da012ab3. Latest consensus state: current_term: 1 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:43.314988 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:43.318038 31900 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } attempt: 7
I20260809 14:29:43.323299 32033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:43.324124 32033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:43.327539 32035 catalog_manager.cc:1347] Generated new cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:29:43.327833 32035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 14:29:43.329752 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:43.330713 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:43.362046 32035 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 14:29:43.368248 32035 catalog_manager.cc:1504] Loading token signing keys...
I20260809 14:29:43.418121 31900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e: Bootstrap starting.
I20260809 14:29:43.443637 31900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:43.446147 31900 log.cc:824] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e: Log is configured to *not* fsync() on all Append() calls
I20260809 14:29:43.449713 32035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Generated new TSK 0
I20260809 14:29:43.454252 32035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 14:29:43.458005 31900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e: No bootstrap required, opened a new log
I20260809 14:29:43.509048 31900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:43.510001 31900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:43.510351 31900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852dbd0962ae4211923851c7a632768e, State: Initialized, Role: FOLLOWER
I20260809 14:29:43.512061 31729 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
/tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.252.65:0
--local_ip_for_outbound_sockets=127.30.252.65
--webserver_interface=127.30.252.65
--webserver_port=0
--tserver_master_addrs=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--builtin_ntp_servers=127.30.252.84:46315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 14:29:43.511560 31900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:43.517923 31900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 14:29:43.518119 31869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:43.519235 31869 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d10512ac8b3406098fad1f7da012ab3 in term 0.
I20260809 14:29:43.520556 31866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:43.520769 31868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e"
I20260809 14:29:43.516986 32044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:43.522838 32044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:43.528357 31866 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Refusing update from remote peer 9d10512ac8b3406098fad1f7da012ab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 14:29:43.530247 32033 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:29:43.612603 32044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d10512ac8b3406098fad1f7da012ab3. Latest consensus state: current_term: 1 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:43.612751 32053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:43.613473 32044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:43.613708 32053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:43.663918 32062 catalog_manager.cc:1259] Loaded cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:29:43.664201 32062 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e: loading cluster ID for follower catalog manager: success
I20260809 14:29:43.671495 32062 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e: acquiring CA information for follower catalog manager: success
I20260809 14:29:43.677807 32062 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 14:29:43.946825 32031 catalog_manager.cc:1259] Loaded cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:29:43.947201 32031 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: loading cluster ID for follower catalog manager: success
I20260809 14:29:43.954859 32031 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: acquiring CA information for follower catalog manager: success
I20260809 14:29:43.985698 32031 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 14:29:44.035849 32053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:44.036370 32053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:44.038789 32053 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:44.039935 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:44.041019 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:44.043211 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:44.044090 32044 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:44.112701 32033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:44.113626 32033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: LEADER
W20260809 14:29:44.246065 32045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 14:29:44.256775 32045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 14:29:44.258380 32045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 14:29:44.333940 32045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 14:29:44.335615 32045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.252.65
I20260809 14:29:44.402098 32045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.252.84:46315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.252.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.252.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.252.65
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 13:36:37 UTC on ccycloud.quasar-iqymvz.root.comops.site
build id 4576526
TSAN enabled
I20260809 14:29:44.404117 32045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 14:29:44.406563 32045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 14:29:44.471514 32073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:44.626155 32079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:44.628928 32079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:44.634155 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:44.634148 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:44.636377 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:44.641568 32081 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:44.633113 32079 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:45.190460 32084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:45.194211 32084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:45.197247 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:45.199321 32084 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:45.200407 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:45.201905 31842 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:45.205979 32084 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:45.575798 32085 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:45.578186 32085 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:45.585076 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:45.590190 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:45.591996 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee93a411a1b04d38998018f8c6be59e0; no voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:45.595304 32085 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:45.597254 32085 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:45.789258 32087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:45.789872 32087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:45.793251 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:45.795632 32087 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:45.796625 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:45.798084 31842 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:45.798857 32087 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:46.193164 32088 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:46.194140 32088 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:46.198591 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:46.203424 32088 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:46.203462 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:46.204864 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee93a411a1b04d38998018f8c6be59e0; no voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:46.211231 32088 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:29:44.471552 32074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 14:29:45.846022 32071 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 32045
W20260809 14:29:46.924259 32071 kernel_stack_watchdog.cc:198] Thread 32045 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 377ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260809 14:29:46.926303 32045 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 2.462s user 0.411s sys 0.777s
W20260809 14:29:46.926681 32045 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 2.462s user 0.411s sys 0.777s
W20260809 14:29:46.927660 32076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 14:29:46.929548 32075 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 2451 milliseconds
I20260809 14:29:46.929595 32045 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 14:29:46.931069 32045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 14:29:46.933633 32045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 14:29:46.935088 32045 hybrid_clock.cc:648] HybridClock initialized: now 1786285786935050 us; error 38 us; skew 500 ppm
I20260809 14:29:46.936121 32045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 14:29:46.949649 32045 webserver.cc:460] Webserver started at http://127.30.252.65:35719/ using document root <none> and password file <none>
I20260809 14:29:46.950850 32045 fs_manager.cc:359] Metadata directory not provided
I20260809 14:29:46.951117 32045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 14:29:46.951653 32045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 14:29:46.958628 32045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/data/instance:
uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
format_stamp: "Formatted at 2026-08-09 14:29:46 on dist-test-slave-10pc"
I20260809 14:29:46.960171 32045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/instance:
uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
format_stamp: "Formatted at 2026-08-09 14:29:46 on dist-test-slave-10pc"
I20260809 14:29:46.970280 32045 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260809 14:29:46.978020 32094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:46.979373 32045 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260809 14:29:46.979750 32045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal
uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
format_stamp: "Formatted at 2026-08-09 14:29:46 on dist-test-slave-10pc"
I20260809 14:29:46.980171 32045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 14:29:47.041910 32045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 14:29:47.043924 32045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 14:29:47.044480 32045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 14:29:47.047895 32045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 14:29:47.053632 32045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 14:29:47.053900 32045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:47.054179 32045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 14:29:47.054375 32045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:47.125627 32149 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:47.126071 32149 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:47.129344 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:47.130479 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:47.131227 32149 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:47.132509 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee93a411a1b04d38998018f8c6be59e0; no voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:47.133553 32149 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:47.189996 32191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:47.190574 32191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:47.193452 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:47.195331 32191 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:47.196621 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:47.198184 31842 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:47.199210 32191 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:47.242623 32045 rpc_server.cc:307] RPC server started. Bound to: 127.30.252.65:41831
I20260809 14:29:47.242743 32221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.252.65:41831 every 8 connection(s)
I20260809 14:29:47.245625 32045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/data/info.pb
I20260809 14:29:47.256048 31729 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu as pid 32045
I20260809 14:29:47.256666 31729 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/instance
I20260809 14:29:47.274267 31729 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
/tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.252.66:0
--local_ip_for_outbound_sockets=127.30.252.66
--webserver_interface=127.30.252.66
--webserver_port=0
--tserver_master_addrs=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--builtin_ntp_servers=127.30.252.84:46315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 14:29:47.307523 32223 heartbeater.cc:347] Connected to a master server at 127.30.252.124:34903
I20260809 14:29:47.307682 32222 heartbeater.cc:347] Connected to a master server at 127.30.252.125:42169
I20260809 14:29:47.308017 32223 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:47.308202 32222 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:47.309345 32223 heartbeater.cc:511] Master 127.30.252.124:34903 requested a full tablet report, sending...
I20260809 14:29:47.309419 32222 heartbeater.cc:511] Master 127.30.252.125:42169 requested a full tablet report, sending...
I20260809 14:29:47.312206 32224 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:29:47.312574 32224 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:47.313082 31858 ts_manager.cc:194] Registered new tserver with Master: e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:47.313432 32224 heartbeater.cc:511] Master 127.30.252.126:43223 requested a full tablet report, sending...
I20260809 14:29:47.315764 31774 ts_manager.cc:194] Registered new tserver with Master: e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:47.315866 31948 ts_manager.cc:194] Registered new tserver with Master: e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:47.318449 31774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.252.65:59533
W20260809 14:29:47.634821 32234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 14:29:47.635448 32234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 14:29:47.636081 32234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 14:29:47.686455 32234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 14:29:47.687439 32234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.252.66
I20260809 14:29:47.731330 32234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.252.84:46315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.252.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.252.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.252.66
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 13:36:37 UTC on ccycloud.quasar-iqymvz.root.comops.site
build id 4576526
TSAN enabled
I20260809 14:29:47.732860 32234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 14:29:47.734452 32234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 14:29:47.755355 32242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:47.777849 32249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:47.778323 32249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:47.780299 32249 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:47.782743 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:47.782696 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:47.785465 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:47.786520 32249 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:47.822193 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:29:48.055204 32250 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:48.055639 32250 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:48.057512 32250 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:48.058792 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:48.059021 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:48.061005 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee93a411a1b04d38998018f8c6be59e0; no voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:48.062023 32250 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:48.631458 32252 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:48.631886 32252 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:48.633566 32252 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:48.634478 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:48.635286 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:48.637251 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee93a411a1b04d38998018f8c6be59e0; no voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:48.637864 32252 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:29:47.756242 32243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:47.758574 32234 server_base.cc:1034] running on GCE node
W20260809 14:29:47.756579 32246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:49.008682 32234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 14:29:49.011607 32234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 14:29:49.013052 32234 hybrid_clock.cc:648] HybridClock initialized: now 1786285789012987 us; error 70 us; skew 500 ppm
I20260809 14:29:49.013840 32234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 14:29:49.020444 32234 webserver.cc:460] Webserver started at http://127.30.252.66:44629/ using document root <none> and password file <none>
I20260809 14:29:49.021426 32234 fs_manager.cc:359] Metadata directory not provided
I20260809 14:29:49.021627 32234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 14:29:49.022029 32234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 14:29:49.026918 32234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/data/instance:
uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
format_stamp: "Formatted at 2026-08-09 14:29:49 on dist-test-slave-10pc"
I20260809 14:29:49.027963 32234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/instance:
uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
format_stamp: "Formatted at 2026-08-09 14:29:49 on dist-test-slave-10pc"
I20260809 14:29:49.035075 32234 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260809 14:29:49.040596 32259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:49.041611 32234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 14:29:49.041953 32234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal
uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
format_stamp: "Formatted at 2026-08-09 14:29:49 on dist-test-slave-10pc"
I20260809 14:29:49.042312 32234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 14:29:49.099463 32234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 14:29:49.101037 32234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 14:29:49.101508 32234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 14:29:49.104106 32234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 14:29:49.108346 32234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 14:29:49.108610 32234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 14:29:49.108886 32234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 14:29:49.109077 32234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:49.193492 32328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:49.194267 32328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:49.197654 32328 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:49.197834 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:49.199223 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:49.200738 31842 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:49.201841 32328 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:29:49.218691 32344 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:49.219444 32344 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:49.222877 32344 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:49.223043 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:49.223140 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:49.223855 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee93a411a1b04d38998018f8c6be59e0 in term 1.
I20260809 14:29:49.225248 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0; no voters: 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:49.226148 32344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:29:49.226491 32344 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:49.226821 32344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:29:49.232980 32344 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:49.234798 32344 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 election: Requested vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:49.236007 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3"
I20260809 14:29:49.236711 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e"
I20260809 14:29:49.237305 31866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:29:49.245748 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee93a411a1b04d38998018f8c6be59e0 in term 2.
I20260809 14:29:49.247085 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0; no voters: 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:49.248263 32344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 14:29:49.252043 32344 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 2 LEADER]: Becoming Leader. State: Replica: ee93a411a1b04d38998018f8c6be59e0, State: Running, Role: LEADER
I20260809 14:29:49.253513 32344 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:49.266036 32366 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee93a411a1b04d38998018f8c6be59e0. Latest consensus state: current_term: 2 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:49.267503 32366 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:49.273444 31866 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 2 FOLLOWER]: Rejecting Update request from peer 9d10512ac8b3406098fad1f7da012ab3 for earlier term 1. Current term is 2. Ops: []
I20260809 14:29:49.275731 32373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 14:29:49.276418 32371 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260809 14:29:49.281468 32375 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 14:29:49.282299 32375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d10512ac8b3406098fad1f7da012ab3, State: Running, Role: LEADER
I20260809 14:29:49.284349 32375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:49.286903 32375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:29:49.290117 32373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 14:29:49.297550 32373 catalog_manager.cc:1259] Loaded cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:29:49.297966 32373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 14:29:49.306759 32373 catalog_manager.cc:1504] Loading token signing keys...
I20260809 14:29:49.313778 32373 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Loaded TSK: 0
I20260809 14:29:49.316727 32373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 14:29:49.337800 32234 rpc_server.cc:307] RPC server started. Bound to: 127.30.252.66:40891
I20260809 14:29:49.337929 32388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.252.66:40891 every 8 connection(s)
I20260809 14:29:49.341071 32234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/data/info.pb
I20260809 14:29:49.342784 31729 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu as pid 32234
I20260809 14:29:49.343772 31729 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/instance
I20260809 14:29:49.359230 31729 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
/tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.252.67:0
--local_ip_for_outbound_sockets=127.30.252.67
--webserver_interface=127.30.252.67
--webserver_port=0
--tserver_master_addrs=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--builtin_ntp_servers=127.30.252.84:46315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260809 14:29:49.420871 32391 heartbeater.cc:347] Connected to a master server at 127.30.252.124:34903
I20260809 14:29:49.421511 32391 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:49.422653 32389 heartbeater.cc:347] Connected to a master server at 127.30.252.125:42169
I20260809 14:29:49.423153 32389 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:49.423228 32391 heartbeater.cc:511] Master 127.30.252.124:34903 requested a full tablet report, sending...
I20260809 14:29:49.424062 32389 heartbeater.cc:511] Master 127.30.252.125:42169 requested a full tablet report, sending...
I20260809 14:29:49.426252 31948 ts_manager.cc:194] Registered new tserver with Master: 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:49.426663 31858 ts_manager.cc:194] Registered new tserver with Master: 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:49.429863 31948 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.252.66:37847
I20260809 14:29:49.435715 32396 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:29:49.436322 32396 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:49.437590 32396 heartbeater.cc:511] Master 127.30.252.126:43223 requested a full tablet report, sending...
I20260809 14:29:49.440591 31774 ts_manager.cc:194] Registered new tserver with Master: 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:29:49.725087 32397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 14:29:49.725637 32397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 14:29:49.726244 32397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 14:29:49.766733 32397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 14:29:49.767722 32397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.252.67
I20260809 14:29:49.798851 31866 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 2 FOLLOWER]: Refusing update from remote peer ee93a411a1b04d38998018f8c6be59e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 14:29:49.800364 32407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 14:29:49.805977 32408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee93a411a1b04d38998018f8c6be59e0. Latest consensus state: current_term: 2 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:49.807384 31785 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 2 FOLLOWER]: Refusing update from remote peer ee93a411a1b04d38998018f8c6be59e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 14:29:49.807075 32408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:49.807757 32397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.252.84:46315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.252.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.252.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.252.67
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 13:36:37 UTC on ccycloud.quasar-iqymvz.root.comops.site
build id 4576526
TSAN enabled
I20260809 14:29:49.809384 32397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 14:29:49.809465 32407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 14:29:49.811947 32397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 14:29:49.825114 32411 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:49.826539 32408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:49.827661 32408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:49.828210 32411 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:49.835920 32223 heartbeater.cc:503] Master 127.30.252.124:34903 was elected leader, sending a full tablet report...
I20260809 14:29:49.845976 32425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:49.847108 32425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:49.847549 32419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee93a411a1b04d38998018f8c6be59e0. Latest consensus state: current_term: 2 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:49.848629 32419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 14:29:49.852090 32418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:49.921216 32014 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 14:29:49.937579 32391 heartbeater.cc:503] Master 127.30.252.124:34903 was elected leader, sending a full tablet report...
I20260809 14:29:50.313037 32408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:29:50.313520 32408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.315150 32408 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:50.316233 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:50.316668 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:50.318405 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:50.319640 32408 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 14:29:50.331521 32419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:29:50.332355 32419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.337522 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:50.341272 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:50.342795 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d10512ac8b3406098fad1f7da012ab3 in term 2.
I20260809 14:29:50.350414 32407 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:50.351346 32407 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:50.365460 31760 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3; no voters: ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:50.365406 32419 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:50.366550 32419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 14:29:50.367168 32419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:29:50.367733 32419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 14:29:50.379242 32419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.383064 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0"
I20260809 14:29:50.382949 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e"
I20260809 14:29:50.382098 32419 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 3 election: Requested vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:50.383741 31866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 2 FOLLOWER]: Advancing to term 3
I20260809 14:29:50.395309 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d10512ac8b3406098fad1f7da012ab3 in term 3.
I20260809 14:29:50.396740 31760 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3; no voters: ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:50.397505 32419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 14:29:50.398132 32419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 3 LEADER]: Becoming Leader. State: Replica: 9d10512ac8b3406098fad1f7da012ab3, State: Running, Role: LEADER
I20260809 14:29:50.399315 32419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.402017 31866 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 3 FOLLOWER]: Rejecting Update request from peer ee93a411a1b04d38998018f8c6be59e0 for earlier term 2. Current term is 3. Ops: []
I20260809 14:29:50.403419 32407 consensus_queue.cc:1046] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 14:29:50.404495 32428 raft_consensus.cc:3032] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 14:29:50.404783 32428 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee93a411a1b04d38998018f8c6be59e0, State: Running, Role: LEADER
I20260809 14:29:50.404062 32430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d10512ac8b3406098fad1f7da012ab3. Latest consensus state: current_term: 3 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:50.405246 32430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:50.405457 32428 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.406710 32428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 2 FOLLOWER]: Advancing to term 3
I20260809 14:29:50.407106 32431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 14:29:50.414563 32431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 14:29:50.417294 32431 catalog_manager.cc:1259] Loaded cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:29:50.417496 32431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 14:29:50.423655 32431 catalog_manager.cc:1504] Loading token signing keys...
I20260809 14:29:50.428140 32431 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Loaded TSK: 0
I20260809 14:29:50.429939 32431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 14:29:50.835767 32428 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:50.836203 32428 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.837975 32428 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:50.839030 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:50.839275 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:29:50.839663 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee93a411a1b04d38998018f8c6be59e0 in term 3.
I20260809 14:29:50.840744 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0; no voters: 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:50.841351 32428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 14:29:50.841668 32428 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:50.841921 32428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 3 FOLLOWER]: Advancing to term 4
I20260809 14:29:50.846910 32428 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.849258 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e"
I20260809 14:29:50.849774 31866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 3 FOLLOWER]: Advancing to term 4
I20260809 14:29:50.849610 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3"
I20260809 14:29:50.848270 32428 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 4 election: Requested vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:50.855046 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee93a411a1b04d38998018f8c6be59e0 in term 4.
I20260809 14:29:50.855944 31868 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 4 FOLLOWER]: Rejecting Update request from peer 9d10512ac8b3406098fad1f7da012ab3 for earlier term 3. Current term is 4. Ops: []
I20260809 14:29:50.856114 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0; no voters: 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:50.857182 32428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 14:29:50.857730 32428 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 4 LEADER]: Becoming Leader. State: Replica: ee93a411a1b04d38998018f8c6be59e0, State: Running, Role: LEADER
I20260809 14:29:50.858599 32428 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.860709 32430 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 14:29:50.861582 32430 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 14:29:50.861959 32430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9d10512ac8b3406098fad1f7da012ab3, State: Running, Role: LEADER
I20260809 14:29:50.862911 32430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:50.867413 32430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 14:29:50.871197 32407 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee93a411a1b04d38998018f8c6be59e0. Latest consensus state: current_term: 4 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:50.871945 32407 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:50.874008 32433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 14:29:50.878841 32433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 14:29:50.882979 32433 catalog_manager.cc:1259] Loaded cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:29:50.883244 32433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 14:29:50.888499 32433 catalog_manager.cc:1504] Loading token signing keys...
I20260809 14:29:50.891988 32433 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Loaded TSK: 0
I20260809 14:29:50.893590 32433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 14:29:50.950316 32396 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:29:49.852857 32397 server_base.cc:1034] running on GCE node
W20260809 14:29:49.853404 32424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 14:29:49.852101 32420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:51.186571 32397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 14:29:51.191347 32397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 14:29:51.192853 32397 hybrid_clock.cc:648] HybridClock initialized: now 1786285791192796 us; error 62 us; skew 500 ppm
I20260809 14:29:51.193784 32397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 14:29:51.207198 32397 webserver.cc:460] Webserver started at http://127.30.252.67:43999/ using document root <none> and password file <none>
I20260809 14:29:51.208177 32397 fs_manager.cc:359] Metadata directory not provided
I20260809 14:29:51.208469 32397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 14:29:51.208985 32397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 14:29:51.214251 32397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/data/instance:
uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
format_stamp: "Formatted at 2026-08-09 14:29:51 on dist-test-slave-10pc"
I20260809 14:29:51.215377 32397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/instance:
uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
format_stamp: "Formatted at 2026-08-09 14:29:51 on dist-test-slave-10pc"
I20260809 14:29:51.225232 32397 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.011s sys 0.000s
I20260809 14:29:51.234125 32438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:51.235203 32397 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.004s
I20260809 14:29:51.235605 32397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal
uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
format_stamp: "Formatted at 2026-08-09 14:29:51 on dist-test-slave-10pc"
I20260809 14:29:51.235981 32397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 14:29:51.299762 32397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 14:29:51.301263 32397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 14:29:51.301692 32397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 14:29:51.304195 32397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 14:29:51.308825 32397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 14:29:51.309026 32397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:51.309242 32397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 14:29:51.309374 32397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 14:29:51.344771 32430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:51.345412 32430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:51.347528 32430 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:51.350039 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:51.350075 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:29:51.350773 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d10512ac8b3406098fad1f7da012ab3 in term 4.
I20260809 14:29:51.352202 31760 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3; no voters:
I20260809 14:29:51.353199 32430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 14:29:51.353636 32430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:51.353971 32430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 14:29:51.360342 32430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:51.362557 32430 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 5 election: Requested vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:29:51.363026 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e"
I20260809 14:29:51.363781 31866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 4 FOLLOWER]: Advancing to term 5
I20260809 14:29:51.363759 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0"
I20260809 14:29:51.374574 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d10512ac8b3406098fad1f7da012ab3 in term 5.
I20260809 14:29:51.377167 31760 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3; no voters: ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:51.378361 32430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 14:29:51.380242 32430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 5 LEADER]: Becoming Leader. State: Replica: 9d10512ac8b3406098fad1f7da012ab3, State: Running, Role: LEADER
I20260809 14:29:51.381044 32430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:51.385339 32488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d10512ac8b3406098fad1f7da012ab3. Latest consensus state: current_term: 5 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:51.386158 32488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:51.387982 32495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 14:29:51.395875 32495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 14:29:51.399708 32495 catalog_manager.cc:1259] Loaded cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:29:51.400017 32495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 14:29:51.405545 32495 catalog_manager.cc:1504] Loading token signing keys...
I20260809 14:29:51.409528 32495 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Loaded TSK: 0
I20260809 14:29:51.410504 31866 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 5 FOLLOWER]: Rejecting Update request from peer ee93a411a1b04d38998018f8c6be59e0 for earlier term 4. Current term is 5. Ops: []
I20260809 14:29:51.410769 32495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 14:29:51.412726 32511 consensus_queue.cc:1046] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 14:29:51.416296 32516 raft_consensus.cc:3032] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 14:29:51.416780 32516 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ee93a411a1b04d38998018f8c6be59e0, State: Running, Role: LEADER
I20260809 14:29:51.417912 32516 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:51.419626 32516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 4 FOLLOWER]: Advancing to term 5
I20260809 14:29:51.450768 32391 heartbeater.cc:503] Master 127.30.252.124:34903 was elected leader, sending a full tablet report...
I20260809 14:29:51.553937 32397 rpc_server.cc:307] RPC server started. Bound to: 127.30.252.67:44797
I20260809 14:29:51.554301 32564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.252.67:44797 every 8 connection(s)
I20260809 14:29:51.557205 32397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/data/info.pb
I20260809 14:29:51.566857 31729 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu as pid 32397
I20260809 14:29:51.567548 31729 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/instance
I20260809 14:29:51.640079 32565 heartbeater.cc:347] Connected to a master server at 127.30.252.125:42169
I20260809 14:29:51.640882 32565 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:51.643487 32567 heartbeater.cc:347] Connected to a master server at 127.30.252.124:34903
I20260809 14:29:51.644016 32569 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:29:51.644178 32567 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:51.644495 32569 heartbeater.cc:464] Registering TS with master...
I20260809 14:29:51.645795 32565 heartbeater.cc:511] Master 127.30.252.125:42169 requested a full tablet report, sending...
I20260809 14:29:51.646077 32569 heartbeater.cc:511] Master 127.30.252.126:43223 requested a full tablet report, sending...
I20260809 14:29:51.646262 32567 heartbeater.cc:511] Master 127.30.252.124:34903 requested a full tablet report, sending...
I20260809 14:29:51.649384 31858 ts_manager.cc:194] Registered new tserver with Master: 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:51.649847 31948 ts_manager.cc:194] Registered new tserver with Master: 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:51.650198 31774 ts_manager.cc:194] Registered new tserver with Master: 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:51.652209 31774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.252.67:36831
I20260809 14:29:51.668999 31729 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 14:29:51.778791 31774 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45650:
name: "default.Table_f4a40a5b9dd74b979cde711f5d4d2873"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:51.786164 31774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:51.788969 31773 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45650:
name: "default.Table_8eb3e7febb5e45279fa895cce75a6c54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:51.791782 31773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8eb3e7febb5e45279fa895cce75a6c54 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:51.795795 31771 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45650:
name: "default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:51.799067 31771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:51.801931 31959 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 5 FOLLOWER]: Refusing update from remote peer 9d10512ac8b3406098fad1f7da012ab3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 14:29:51.807153 31770 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45650:
name: "default.Table_e2e898418bd240bbba124f855dfd4bb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:51.810338 31770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2e898418bd240bbba124f855dfd4bb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:51.812559 32488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 14:29:51.813228 32488 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Peer ee93a411a1b04d38998018f8c6be59e0 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 14:29:51.820996 31959 pending_rounds.cc:76] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Aborting all ops after (but not including) 5
I20260809 14:29:51.821573 31959 pending_rounds.cc:97] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260809 14:29:51.822368 31959 raft_consensus.cc:2866] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 14:29:51.832106 32516 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d10512ac8b3406098fad1f7da012ab3. Latest consensus state: current_term: 5 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:51.833755 32516 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:51.842017 31866 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 5 FOLLOWER]: Refusing update from remote peer 9d10512ac8b3406098fad1f7da012ab3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260809 14:29:51.844764 32430 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 14:29:51.853978 32488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:51.855252 32488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:51.868795 32516 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:51.868618 32511 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:51.869647 32516 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:51.869796 32511 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:51.879428 32223 heartbeater.cc:503] Master 127.30.252.124:34903 was elected leader, sending a full tablet report...
I20260809 14:29:51.958549 31770 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45650:
table { table_name: "default.Table_e2e898418bd240bbba124f855dfd4bb1" } new_table_name: "default.Table_5407d24d582f49c59ad48b0e2d5d9a4b" modify_external_catalogs: true
I20260809 14:29:51.961656 31773 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45650:
table { table_name: "default.Table_8eb3e7febb5e45279fa895cce75a6c54" } new_table_name: "default.Table_9421cfe39abc4d408a26df8838273d64" modify_external_catalogs: true
I20260809 14:29:51.961975 32396 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:29:51.986053 31729 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu with pid 31738
I20260809 14:29:51.991853 32605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:51.993111 32605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:51.993423 32596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d10512ac8b3406098fad1f7da012ab3. Latest consensus state: current_term: 5 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:51.994313 32596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:51.995183 32608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "9d10512ac8b3406098fad1f7da012ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:51.996276 32608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
W20260809 14:29:52.022912 31840 connection.cc:535] server connection from 127.0.0.1:48012 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 14:29:52.023154 31931 connection.cc:535] server connection from 127.0.0.1:43806 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 14:29:52.157864 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
W20260809 14:29:52.162506 32569 heartbeater.cc:644] Failed to heartbeat to 127.30.252.126:43223 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
I20260809 14:29:52.361814 32608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:52.362342 32608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:52.364327 32608 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
W20260809 14:29:52.366227 31840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260809 14:29:52.366063 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
W20260809 14:29:52.369617 31840 leader_election.cc:334] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
I20260809 14:29:52.369993 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:29:52.370594 32608 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 14:29:52.394948 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
W20260809 14:29:52.399401 32224 heartbeater.cc:644] Failed to heartbeat to 127.30.252.126:43223 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
I20260809 14:29:52.407590 32620 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:52.407995 32620 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:52.409607 32620 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:52.410737 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:29:52.411383 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee93a411a1b04d38998018f8c6be59e0 in term 5.
W20260809 14:29:52.412300 31930 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
I20260809 14:29:52.412581 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0; no voters:
I20260809 14:29:52.413302 32620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 14:29:52.413575 32620 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9d10512ac8b3406098fad1f7da012ab3)
I20260809 14:29:52.413959 32620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 5 FOLLOWER]: Advancing to term 6
W20260809 14:29:52.416774 31930 leader_election.cc:334] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
I20260809 14:29:52.420656 32620 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:52.422292 32620 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 6 election: Requested vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:29:52.423343 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e"
I20260809 14:29:52.423965 31866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 5 FOLLOWER]: Advancing to term 6
W20260809 14:29:52.425968 31930 leader_election.cc:334] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
I20260809 14:29:52.429553 31866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee93a411a1b04d38998018f8c6be59e0 in term 6.
I20260809 14:29:52.430543 31933 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0; no voters: 9d10512ac8b3406098fad1f7da012ab3
I20260809 14:29:52.431216 32620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 14:29:52.432442 32620 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 6 LEADER]: Becoming Leader. State: Replica: ee93a411a1b04d38998018f8c6be59e0, State: Running, Role: LEADER
I20260809 14:29:52.433131 32620 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 5.11, Last appended by leader: 11, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:29:52.436233 32624 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee93a411a1b04d38998018f8c6be59e0. Latest consensus state: current_term: 6 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:52.437314 32624 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:52.440246 32625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 14:29:52.447713 32625 catalog_manager.cc:669] Loaded metadata for table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]
I20260809 14:29:52.449077 32625 catalog_manager.cc:669] Loaded metadata for table default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 [id=5504fd425a0b4fb4bcd5acb5d506475f]
I20260809 14:29:52.450268 32625 catalog_manager.cc:669] Loaded metadata for table default.Table_8eb3e7febb5e45279fa895cce75a6c54 [id=a53a4159cb0148e0b87cef9a21aae33c]
I20260809 14:29:52.451344 32625 catalog_manager.cc:669] Loaded metadata for table default.Table_e2e898418bd240bbba124f855dfd4bb1 [id=f54dffd569df4cd8a21205544a7a12b7]
I20260809 14:29:52.455626 32625 tablet_loader.cc:96] loaded metadata for tablet 446c935f59874c85b29700511b23da46 (table default.Table_8eb3e7febb5e45279fa895cce75a6c54 [id=a53a4159cb0148e0b87cef9a21aae33c])
I20260809 14:29:52.456688 32625 tablet_loader.cc:96] loaded metadata for tablet 4759d2711e52406ea0ea262e395d686f (table default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 [id=5504fd425a0b4fb4bcd5acb5d506475f])
I20260809 14:29:52.457755 32625 tablet_loader.cc:96] loaded metadata for tablet a0cb9f368ba64220b1bece7624b773b7 (table default.Table_e2e898418bd240bbba124f855dfd4bb1 [id=f54dffd569df4cd8a21205544a7a12b7])
I20260809 14:29:52.458570 32625 tablet_loader.cc:96] loaded metadata for tablet b17b6faccdd74299b6c397d7f41689e6 (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021])
I20260809 14:29:52.459461 32625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 14:29:52.462723 32625 catalog_manager.cc:1259] Loaded cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:29:52.462980 32625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 14:29:52.466292 32625 catalog_manager.cc:1504] Loading token signing keys...
I20260809 14:29:52.468966 32625 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Loaded TSK: 0
I20260809 14:29:52.470069 32625 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 14:29:52.475327 32396 heartbeater.cc:644] Failed to heartbeat to 127.30.252.126:43223 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
W20260809 14:29:52.829771 31930 consensus_peers.cc:487] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 -> Peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Couldn't send request to peer 9d10512ac8b3406098fad1f7da012ab3. Status: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 14:29:52.998281 31866 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Refusing update from remote peer ee93a411a1b04d38998018f8c6be59e0: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 12. (index mismatch)
I20260809 14:29:53.000403 32624 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 14:29:53.004746 32391 heartbeater.cc:503] Master 127.30.252.124:34903 was elected leader, sending a full tablet report...
I20260809 14:29:53.032341 32630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee93a411a1b04d38998018f8c6be59e0. Latest consensus state: current_term: 6 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:53.033452 32630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:53.048532 32630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:53.049255 32630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:29:53.050104 32637 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:29:53.050649 32637 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: LEADER
I20260809 14:29:53.081542 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 446c935f59874c85b29700511b23da46 (DEFAULT_TABLE table=default.Table_8eb3e7febb5e45279fa895cce75a6c54 [id=a53a4159cb0148e0b87cef9a21aae33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.083163 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 446c935f59874c85b29700511b23da46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.081542 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 4759d2711e52406ea0ea262e395d686f (DEFAULT_TABLE table=default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 [id=5504fd425a0b4fb4bcd5acb5d506475f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.085721 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4759d2711e52406ea0ea262e395d686f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.087404 32145 tablet_service.cc:1467] Processing CreateTablet for tablet b17b6faccdd74299b6c397d7f41689e6 (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.088555 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b17b6faccdd74299b6c397d7f41689e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.079972 32148 tablet_service.cc:1467] Processing CreateTablet for tablet a0cb9f368ba64220b1bece7624b773b7 (DEFAULT_TABLE table=default.Table_e2e898418bd240bbba124f855dfd4bb1 [id=f54dffd569df4cd8a21205544a7a12b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.090903 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0cb9f368ba64220b1bece7624b773b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.103470 32491 tablet_service.cc:1467] Processing CreateTablet for tablet a0cb9f368ba64220b1bece7624b773b7 (DEFAULT_TABLE table=default.Table_e2e898418bd240bbba124f855dfd4bb1 [id=f54dffd569df4cd8a21205544a7a12b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.103469 32307 tablet_service.cc:1467] Processing CreateTablet for tablet b17b6faccdd74299b6c397d7f41689e6 (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.104293 32310 tablet_service.cc:1467] Processing CreateTablet for tablet a0cb9f368ba64220b1bece7624b773b7 (DEFAULT_TABLE table=default.Table_e2e898418bd240bbba124f855dfd4bb1 [id=f54dffd569df4cd8a21205544a7a12b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.105461 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b17b6faccdd74299b6c397d7f41689e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.105576 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 446c935f59874c85b29700511b23da46 (DEFAULT_TABLE table=default.Table_8eb3e7febb5e45279fa895cce75a6c54 [id=a53a4159cb0148e0b87cef9a21aae33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.103694 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 446c935f59874c85b29700511b23da46 (DEFAULT_TABLE table=default.Table_8eb3e7febb5e45279fa895cce75a6c54 [id=a53a4159cb0148e0b87cef9a21aae33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.106806 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 446c935f59874c85b29700511b23da46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.107188 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 446c935f59874c85b29700511b23da46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.108690 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0cb9f368ba64220b1bece7624b773b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.109323 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 4759d2711e52406ea0ea262e395d686f (DEFAULT_TABLE table=default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 [id=5504fd425a0b4fb4bcd5acb5d506475f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.110008 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 4759d2711e52406ea0ea262e395d686f (DEFAULT_TABLE table=default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 [id=5504fd425a0b4fb4bcd5acb5d506475f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.110363 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4759d2711e52406ea0ea262e395d686f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.111179 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4759d2711e52406ea0ea262e395d686f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.113351 32487 tablet_service.cc:1467] Processing CreateTablet for tablet b17b6faccdd74299b6c397d7f41689e6 (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:53.114462 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b17b6faccdd74299b6c397d7f41689e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.116565 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0cb9f368ba64220b1bece7624b773b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:53.157908 32647 tablet_bootstrap.cc:492] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:53.158764 32567 heartbeater.cc:503] Master 127.30.252.124:34903 was elected leader, sending a full tablet report...
I20260809 14:29:53.165149 32647 tablet_bootstrap.cc:654] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.167505 32647 log.cc:824] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Log is configured to *not* fsync() on all Append() calls
I20260809 14:29:53.195063 32647 tablet_bootstrap.cc:492] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:53.195701 32647 ts_tablet_manager.cc:1397] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.038s user 0.015s sys 0.000s
I20260809 14:29:53.200421 32649 tablet_bootstrap.cc:492] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:53.203482 32651 tablet_bootstrap.cc:492] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:53.208096 32649 tablet_bootstrap.cc:654] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.210611 32649 log.cc:824] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Log is configured to *not* fsync() on all Append() calls
I20260809 14:29:53.211508 32651 tablet_bootstrap.cc:654] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.214205 32651 log.cc:824] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Log is configured to *not* fsync() on all Append() calls
I20260809 14:29:53.228559 32649 tablet_bootstrap.cc:492] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:53.229173 32649 ts_tablet_manager.cc:1397] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.029s user 0.013s sys 0.014s
I20260809 14:29:53.230513 32651 tablet_bootstrap.cc:492] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:53.231065 32651 ts_tablet_manager.cc:1397] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.028s user 0.014s sys 0.010s
I20260809 14:29:53.235448 32647 raft_consensus.cc:348] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.236497 32647 raft_consensus.cc:374] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.236840 32647 raft_consensus.cc:729] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.237963 32647 consensus_queue.cc:260] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.248517 32647 ts_tablet_manager.cc:1428] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.052s user 0.039s sys 0.011s
I20260809 14:29:53.261859 32223 heartbeater.cc:503] Master 127.30.252.124:34903 was elected leader, sending a full tablet report...
I20260809 14:29:53.263433 32647 tablet_bootstrap.cc:492] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:53.266168 32651 raft_consensus.cc:348] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.267184 32651 raft_consensus.cc:374] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.267503 32651 raft_consensus.cc:729] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.268627 32651 consensus_queue.cc:260] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.269610 32649 raft_consensus.cc:348] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.270535 32649 raft_consensus.cc:374] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.270829 32649 raft_consensus.cc:729] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.271817 32649 consensus_queue.cc:260] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.274183 32647 tablet_bootstrap.cc:654] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.288244 32647 tablet_bootstrap.cc:492] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:53.288822 32647 ts_tablet_manager.cc:1397] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.010s
W20260809 14:29:53.288913 32229 tablet.cc:2369] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 14:29:53.292353 32647 raft_consensus.cc:348] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.293677 32647 raft_consensus.cc:374] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.294072 32647 raft_consensus.cc:729] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.294982 32647 consensus_queue.cc:260] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.297768 32647 ts_tablet_manager.cc:1428] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260809 14:29:53.298668 32647 tablet_bootstrap.cc:492] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:53.305392 32647 tablet_bootstrap.cc:654] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.310103 32649 ts_tablet_manager.cc:1428] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.081s user 0.034s sys 0.011s
I20260809 14:29:53.315454 32651 ts_tablet_manager.cc:1428] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.084s user 0.049s sys 0.016s
I20260809 14:29:53.317306 32647 tablet_bootstrap.cc:492] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:53.317775 32647 ts_tablet_manager.cc:1397] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.015s
I20260809 14:29:53.320420 32647 raft_consensus.cc:348] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.321247 32647 raft_consensus.cc:374] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.321558 32647 raft_consensus.cc:729] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.322476 32647 consensus_queue.cc:260] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.323159 32649 tablet_bootstrap.cc:492] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:53.325237 32647 ts_tablet_manager.cc:1428] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260809 14:29:53.326175 32647 tablet_bootstrap.cc:492] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:53.332561 32649 tablet_bootstrap.cc:654] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.333201 32651 tablet_bootstrap.cc:492] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:53.336540 32647 tablet_bootstrap.cc:654] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.339612 32651 tablet_bootstrap.cc:654] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.348649 32647 tablet_bootstrap.cc:492] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:53.349181 32647 ts_tablet_manager.cc:1397] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.023s user 0.015s sys 0.002s
I20260809 14:29:53.352747 32647 raft_consensus.cc:348] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.353794 32647 raft_consensus.cc:374] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.354175 32647 raft_consensus.cc:729] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.354985 32647 consensus_queue.cc:260] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.357957 32647 ts_tablet_manager.cc:1428] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260809 14:29:53.362375 32651 tablet_bootstrap.cc:492] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:53.362847 32651 ts_tablet_manager.cc:1397] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.012s
I20260809 14:29:53.365734 32651 raft_consensus.cc:348] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.366544 32651 raft_consensus.cc:374] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.366858 32651 raft_consensus.cc:729] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:29:53.367878 32573 tablet.cc:2369] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 14:29:53.367569 32651 consensus_queue.cc:260] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.368572 32649 tablet_bootstrap.cc:492] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:53.369082 32649 ts_tablet_manager.cc:1397] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.046s user 0.014s sys 0.000s
I20260809 14:29:53.369560 32651 ts_tablet_manager.cc:1428] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260809 14:29:53.373200 32649 raft_consensus.cc:348] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.375574 32651 tablet_bootstrap.cc:492] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:53.375916 32649 raft_consensus.cc:374] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.376266 32649 raft_consensus.cc:729] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.382411 32651 tablet_bootstrap.cc:654] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.382303 32649 consensus_queue.cc:260] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.387998 32649 ts_tablet_manager.cc:1428] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260809 14:29:53.389129 32649 tablet_bootstrap.cc:492] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
W20260809 14:29:53.391880 32398 tablet.cc:2369] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 14:29:53.403036 32649 tablet_bootstrap.cc:654] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.406576 32654 raft_consensus.cc:482] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:53.407173 32654 raft_consensus.cc:504] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.410316 32654 leader_election.cc:288] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:53.412468 32651 tablet_bootstrap.cc:492] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:53.412904 32651 ts_tablet_manager.cc:1397] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.038s user 0.025s sys 0.008s
I20260809 14:29:53.415356 32651 raft_consensus.cc:348] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.416069 32651 raft_consensus.cc:374] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.416347 32651 raft_consensus.cc:729] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.417037 32651 consensus_queue.cc:260] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.419065 32651 ts_tablet_manager.cc:1428] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 14:29:53.419991 32651 tablet_bootstrap.cc:492] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:53.425643 32651 tablet_bootstrap.cc:654] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.434408 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0cb9f368ba64220b1bece7624b773b7" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:53.435389 32515 raft_consensus.cc:2454] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:53.439604 32095 leader_election.cc:302] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:53.440594 32654 raft_consensus.cc:2781] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:53.440982 32654 raft_consensus.cc:482] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:53.441274 32654 raft_consensus.cc:3037] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:53.448400 32654 raft_consensus.cc:504] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.450560 32654 leader_election.cc:288] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:53.451026 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0cb9f368ba64220b1bece7624b773b7" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:53.451459 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0cb9f368ba64220b1bece7624b773b7" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:53.452029 32515 raft_consensus.cc:3037] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:53.453104 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0cb9f368ba64220b1bece7624b773b7" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
W20260809 14:29:53.453743 32099 leader_election.cc:341] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:29:53.455161 32099 leader_election.cc:341] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:53.461676 32651 tablet_bootstrap.cc:492] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:53.462105 32651 ts_tablet_manager.cc:1397] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.042s user 0.026s sys 0.004s
I20260809 14:29:53.463348 32649 tablet_bootstrap.cc:492] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:53.463868 32649 ts_tablet_manager.cc:1397] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.075s user 0.014s sys 0.001s
I20260809 14:29:53.464608 32651 raft_consensus.cc:348] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.465193 32651 raft_consensus.cc:374] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.465498 32651 raft_consensus.cc:729] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.465543 32515 raft_consensus.cc:2454] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:29:53.466120 32651 consensus_queue.cc:260] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.466809 32095 leader_election.cc:302] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:29:53.466825 32649 raft_consensus.cc:348] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.467576 32649 raft_consensus.cc:374] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.467855 32649 raft_consensus.cc:729] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.468730 32649 consensus_queue.cc:260] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.469538 32654 raft_consensus.cc:2781] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:53.469995 32651 ts_tablet_manager.cc:1428] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260809 14:29:53.471781 32649 ts_tablet_manager.cc:1428] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.008s user 0.002s sys 0.005s
I20260809 14:29:53.472709 32649 tablet_bootstrap.cc:492] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:53.478426 32649 tablet_bootstrap.cc:654] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:53.492604 32654 raft_consensus.cc:686] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:29:53.493750 32654 consensus_queue.cc:237] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:53.513962 32649 tablet_bootstrap.cc:492] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:53.514405 32649 ts_tablet_manager.cc:1397] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.042s user 0.014s sys 0.024s
I20260809 14:29:53.516782 32649 raft_consensus.cc:348] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.517459 32649 raft_consensus.cc:374] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:53.517761 32649 raft_consensus.cc:729] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:53.518458 32649 consensus_queue.cc:260] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.523178 32649 ts_tablet_manager.cc:1428] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.008s user 0.002s sys 0.005s
I20260809 14:29:53.527659 31948 catalog_manager.cc:5495] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65). New cstate: current_term: 1 leader_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } }
I20260809 14:29:53.627275 32654 raft_consensus.cc:482] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:53.627738 32654 raft_consensus.cc:504] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.629427 32654 leader_election.cc:288] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:53.630483 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17b6faccdd74299b6c397d7f41689e6" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:53.630707 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17b6faccdd74299b6c397d7f41689e6" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:53.631294 32334 raft_consensus.cc:2454] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:53.631335 32515 raft_consensus.cc:2454] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:53.632406 32099 leader_election.cc:302] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:53.633285 32654 raft_consensus.cc:2781] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:53.633602 32654 raft_consensus.cc:482] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:53.633884 32654 raft_consensus.cc:3037] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:53.639966 32654 raft_consensus.cc:504] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.641789 32654 leader_election.cc:288] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:53.642663 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17b6faccdd74299b6c397d7f41689e6" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:53.642828 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17b6faccdd74299b6c397d7f41689e6" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:53.643214 32334 raft_consensus.cc:3037] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:53.643357 32515 raft_consensus.cc:3037] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:53.647740 32515 raft_consensus.cc:2454] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:29:53.648636 32095 leader_election.cc:302] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:53.649358 32654 raft_consensus.cc:2781] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:53.650763 32334 raft_consensus.cc:2454] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:29:53.649747 32654 raft_consensus.cc:686] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:29:53.651719 32654 consensus_queue.cc:237] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.661129 31948 catalog_manager.cc:5495] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65). New cstate: current_term: 1 leader_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:29:53.680724 32654 raft_consensus.cc:482] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:53.681214 32654 raft_consensus.cc:504] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.684293 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4759d2711e52406ea0ea262e395d686f" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:53.684770 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4759d2711e52406ea0ea262e395d686f" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:53.684953 32334 raft_consensus.cc:2454] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:53.685433 32515 raft_consensus.cc:2454] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:53.683156 32654 leader_election.cc:288] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:29:52.023305 32576 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.30.252.126:43223)
I20260809 14:29:53.686533 32095 leader_election.cc:302] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
W20260809 14:29:53.686729 32576 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.DeleteTable -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 1.66512e+06us
I20260809 14:29:53.687383 32654 raft_consensus.cc:2781] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:53.687717 32654 raft_consensus.cc:482] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 14:29:53.279640 31734 debug-util.cc:398] Leaking SignalData structure 0x7b080002a0c0 after lost signal to thread 32576
I20260809 14:29:53.688037 32654 raft_consensus.cc:3037] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:29:53.688313 32576 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.30.252.126:43223)
I20260809 14:29:53.687999 31729 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
/tmp/dist-test-taskyetPbO/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.252.126:43223
--webserver_interface=127.30.252.126
--webserver_port=39845
--builtin_ntp_servers=127.30.252.84:46315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260809 14:29:53.694258 32654 raft_consensus.cc:504] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.699254 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4759d2711e52406ea0ea262e395d686f" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:53.699369 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4759d2711e52406ea0ea262e395d686f" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
W20260809 14:29:53.699692 32576 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.252.126:43223)
I20260809 14:29:53.700040 32515 raft_consensus.cc:3037] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:53.700073 32334 raft_consensus.cc:3037] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:53.707621 32334 raft_consensus.cc:2454] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:29:53.709039 32099 leader_election.cc:302] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:53.710429 32667 raft_consensus.cc:2781] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:53.711222 32515 raft_consensus.cc:2454] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:29:53.712796 32654 leader_election.cc:288] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:53.714493 32667 raft_consensus.cc:686] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:29:53.715518 32667 consensus_queue.cc:237] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:53.721637 32576 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.30.252.126:43223)
I20260809 14:29:53.729463 31948 catalog_manager.cc:5495] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65). New cstate: current_term: 1 leader_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:29:53.743156 31946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58" } modify_external_catalogs: true
I20260809 14:29:53.743929 31946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58" } modify_external_catalogs: true
I20260809 14:29:53.746688 31945 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8eb3e7febb5e45279fa895cce75a6c54" } new_table_name: "default.Table_9421cfe39abc4d408a26df8838273d64" modify_external_catalogs: true
I20260809 14:29:53.747382 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e2e898418bd240bbba124f855dfd4bb1" } new_table_name: "default.Table_5407d24d582f49c59ad48b0e2d5d9a4b" modify_external_catalogs: true
I20260809 14:29:53.754480 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f4a40a5b9dd74b979cde711f5d4d2873"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:53.756623 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:53.759162 31944 master_service.cc:946] ReplaceTablet: received request to replace tablet a0cb9f368ba64220b1bece7624b773b7 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:53.760941 31943 master_service.cc:946] ReplaceTablet: received request to replace tablet b17b6faccdd74299b6c397d7f41689e6 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:53.791706 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_146d54f420a5452dbc1fa210a2fc1c75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:53.791868 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_ecb0700154564f4d8ae4bff86a6c6743"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:53.787274 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_8580644785dc4eaab7fefc3d44b66089"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:53.795535 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ecb0700154564f4d8ae4bff86a6c6743 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:53.783344 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_982e1611f4824f7dacffbd8faefd41cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:53.799521 31946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 4759d2711e52406ea0ea262e395d686f
I20260809 14:29:53.804428 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet b17b6faccdd74299b6c397d7f41689e6
I20260809 14:29:53.806888 32146 tablet_service.cc:1514] Processing DeleteTablet for tablet b17b6faccdd74299b6c397d7f41689e6 with delete_type TABLET_DATA_DELETED (replaced by tablet ca2383fa88b441e4bb31bea3aac43f2a) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:53.807530 31943 catalog_manager.cc:6565] ReplaceTablet: tablet b17b6faccdd74299b6c397d7f41689e6 deleted and replaced by tablet ca2383fa88b441e4bb31bea3aac43f2a
W20260809 14:29:53.799830 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_982e1611f4824f7dacffbd8faefd41cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:53.807700 32309 tablet_service.cc:1514] Processing DeleteTablet for tablet b17b6faccdd74299b6c397d7f41689e6 with delete_type TABLET_DATA_DELETED (replaced by tablet ca2383fa88b441e4bb31bea3aac43f2a) from {username='slave'} at 127.0.0.1:47010
W20260809 14:29:53.801222 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_146d54f420a5452dbc1fa210a2fc1c75 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:29:53.796736 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8580644785dc4eaab7fefc3d44b66089 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:53.811286 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet b17b6faccdd74299b6c397d7f41689e6 with delete_type TABLET_DATA_DELETED (replaced by tablet ca2383fa88b441e4bb31bea3aac43f2a) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:53.817981 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 4759d2711e52406ea0ea262e395d686f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:53 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:53.818666 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 4759d2711e52406ea0ea262e395d686f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:53 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:53.821920 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 4759d2711e52406ea0ea262e395d686f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:53 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:53.828768 32676 tablet_replica.cc:330] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:53.830360 32678 tablet_replica.cc:330] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:53.832620 32678 raft_consensus.cc:2229] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:53.833652 32678 raft_consensus.cc:2258] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:53.838356 32678 ts_tablet_manager.cc:1905] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:53.849205 32677 tablet_replica.cc:330] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:53.851294 32677 raft_consensus.cc:2229] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:53.852414 32677 raft_consensus.cc:2258] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:53.875537 32677 ts_tablet_manager.cc:1905] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:53.880965 32676 raft_consensus.cc:2229] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:53.886598 32678 ts_tablet_manager.cc:1918] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:53.889946 32678 log.cc:1196] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/b17b6faccdd74299b6c397d7f41689e6
I20260809 14:29:53.891201 32676 pending_rounds.cc:62] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:29:53.891615 32676 pending_rounds.cc:68] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626610805706752 op_type: NO_OP noop_request { }
I20260809 14:29:53.892323 32676 raft_consensus.cc:2866] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:53.892783 32676 raft_consensus.cc:2258] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:53.900712 32678 ts_tablet_manager.cc:1939] T b17b6faccdd74299b6c397d7f41689e6 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
W20260809 14:29:53.900748 31945 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 446c935f59874c85b29700511b23da46 consensus state has no leader
I20260809 14:29:53.903630 31948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5407d24d582f49c59ad48b0e2d5d9a4b" } modify_external_catalogs: true
I20260809 14:29:53.904361 31948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5407d24d582f49c59ad48b0e2d5d9a4b" } modify_external_catalogs: true
I20260809 14:29:53.910717 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet b17b6faccdd74299b6c397d7f41689e6 (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
I20260809 14:29:53.913575 32678 tablet_replica.cc:330] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:53.915132 32678 raft_consensus.cc:2229] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:53.915887 32678 raft_consensus.cc:2258] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:53.917117 32677 ts_tablet_manager.cc:1918] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:53.917866 32677 log.cc:1196] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/b17b6faccdd74299b6c397d7f41689e6
I20260809 14:29:53.919164 32678 ts_tablet_manager.cc:1905] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:53.920006 32677 ts_tablet_manager.cc:1939] T b17b6faccdd74299b6c397d7f41689e6 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:53.924181 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet b17b6faccdd74299b6c397d7f41689e6 (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
I20260809 14:29:53.925956 32677 tablet_replica.cc:330] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:53.925889 32676 ts_tablet_manager.cc:1905] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:53.930675 32677 raft_consensus.cc:2229] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 14:29:53.931339 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 446c935f59874c85b29700511b23da46 (table default.Table_9421cfe39abc4d408a26df8838273d64 [id=a53a4159cb0148e0b87cef9a21aae33c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 446c935f59874c85b29700511b23da46 consensus state has no leader
I20260809 14:29:53.931556 32334 raft_consensus.cc:3037] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:53.938719 32655 raft_consensus.cc:482] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:53.939488 32655 raft_consensus.cc:504] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:53.942910 32677 raft_consensus.cc:2258] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:53.945976 32677 ts_tablet_manager.cc:1905] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:53.950809 32515 raft_consensus.cc:1261] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Refusing update from remote peer e1f94e0d792c46c9a0b6ab42820da6eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 14:29:53.953576 32334 raft_consensus.cc:1261] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Refusing update from remote peer e1f94e0d792c46c9a0b6ab42820da6eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 14:29:53.956099 32667 consensus_queue.cc:1035] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:29:53.958526 32674 consensus_queue.cc:1035] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:29:53.970402 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet a0cb9f368ba64220b1bece7624b773b7
I20260809 14:29:53.978722 31944 catalog_manager.cc:6565] ReplaceTablet: tablet a0cb9f368ba64220b1bece7624b773b7 deleted and replaced by tablet e98e5d4c2bd54d3f9fa343d9d1dc4050
I20260809 14:29:53.983153 32146 tablet_service.cc:1514] Processing DeleteTablet for tablet a0cb9f368ba64220b1bece7624b773b7 with delete_type TABLET_DATA_DELETED (replaced by tablet e98e5d4c2bd54d3f9fa343d9d1dc4050) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:53.987979 32309 tablet_service.cc:1514] Processing DeleteTablet for tablet a0cb9f368ba64220b1bece7624b773b7 with delete_type TABLET_DATA_DELETED (replaced by tablet e98e5d4c2bd54d3f9fa343d9d1dc4050) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:53.988075 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet a0cb9f368ba64220b1bece7624b773b7 with delete_type TABLET_DATA_DELETED (replaced by tablet e98e5d4c2bd54d3f9fa343d9d1dc4050) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:53.990476 32655 leader_election.cc:288] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:54.015226 32677 ts_tablet_manager.cc:1918] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:54.015656 32677 log.cc:1196] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/4759d2711e52406ea0ea262e395d686f
I20260809 14:29:54.016510 32677 ts_tablet_manager.cc:1939] T 4759d2711e52406ea0ea262e395d686f P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:54.021152 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet b17b6faccdd74299b6c397d7f41689e6 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:54.034834 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9421cfe39abc4d408a26df8838273d64" } modify_external_catalogs: true
I20260809 14:29:54.035883 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9421cfe39abc4d408a26df8838273d64" } modify_external_catalogs: true
I20260809 14:29:54.036696 32677 tablet_replica.cc:330] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:54.037014 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 4759d2711e52406ea0ea262e395d686f (table default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 [id=5504fd425a0b4fb4bcd5acb5d506475f]) successfully deleted
I20260809 14:29:54.057991 32678 ts_tablet_manager.cc:1918] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:54.058396 32678 log.cc:1196] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/4759d2711e52406ea0ea262e395d686f
I20260809 14:29:54.059058 32678 ts_tablet_manager.cc:1939] T 4759d2711e52406ea0ea262e395d686f P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:54.067286 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 4759d2711e52406ea0ea262e395d686f (table default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 [id=5504fd425a0b4fb4bcd5acb5d506475f]) successfully deleted
I20260809 14:29:54.068969 32678 tablet_replica.cc:330] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:54.082310 32146 tablet_service.cc:1467] Processing CreateTablet for tablet ca2383fa88b441e4bb31bea3aac43f2a (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.083750 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca2383fa88b441e4bb31bea3aac43f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.085901 32309 tablet_service.cc:1467] Processing CreateTablet for tablet ca2383fa88b441e4bb31bea3aac43f2a (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.087630 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca2383fa88b441e4bb31bea3aac43f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.088857 32656 raft_consensus.cc:482] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:54.089071 32491 tablet_service.cc:1467] Processing CreateTablet for tablet ca2383fa88b441e4bb31bea3aac43f2a (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.089304 32656 raft_consensus.cc:504] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.090113 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca2383fa88b441e4bb31bea3aac43f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.108196 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "446c935f59874c85b29700511b23da46" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:54.110780 32656 leader_election.cc:288] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:54.114614 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_146d54f420a5452dbc1fa210a2fc1c75" } new_table_name: "default.Table_09f478730b6647b8aad8c1632eb94910" modify_external_catalogs: true
I20260809 14:29:54.110004 32170 raft_consensus.cc:2454] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:54.124914 32439 leader_election.cc:302] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:54.130525 32655 raft_consensus.cc:2781] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:54.131110 32655 raft_consensus.cc:482] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:54.131664 32655 raft_consensus.cc:3037] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:54.162951 32691 tablet_bootstrap.cc:492] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:54.173111 32677 raft_consensus.cc:2229] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:54.184437 32677 pending_rounds.cc:62] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 2 pending ops.
I20260809 14:29:54.189371 32677 pending_rounds.cc:68] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316626611915788288 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "a0cb9f368ba64220b1bece7624b773b7" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_5407d24d582f49c59ad48b0e2d5d9a4b" dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" new_extra_config { } }
I20260809 14:29:54.194568 32677 pending_rounds.cc:68] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626610178732032 op_type: NO_OP noop_request { }
I20260809 14:29:54.195166 32677 raft_consensus.cc:2866] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:54.196070 32677 raft_consensus.cc:2258] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:54.197156 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ecb0700154564f4d8ae4bff86a6c6743" } new_table_name: "default.Table_1a5aedd761b04e78a73395ab7562c757" modify_external_catalogs: true
I20260809 14:29:54.206598 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "446c935f59874c85b29700511b23da46" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:54.213725 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "446c935f59874c85b29700511b23da46" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:54.215098 32333 raft_consensus.cc:2454] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:54.227070 32691 tablet_bootstrap.cc:654] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:54.230608 32693 tablet_bootstrap.cc:492] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:54.237458 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_11c018a7f05149e18f070bf70b2f1bcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:54.242455 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11c018a7f05149e18f070bf70b2f1bcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:54.247217 32678 raft_consensus.cc:2229] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:54.248144 32693 tablet_bootstrap.cc:654] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:54.254479 32678 pending_rounds.cc:62] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Trying to abort 2 pending ops.
I20260809 14:29:54.254849 32678 pending_rounds.cc:68] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316626611915788288 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "a0cb9f368ba64220b1bece7624b773b7" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_5407d24d582f49c59ad48b0e2d5d9a4b" dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" new_extra_config { } }
I20260809 14:29:54.246050 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e768d5aab10146f3be037da4134ae39d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:54.259413 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e768d5aab10146f3be037da4134ae39d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:54.263856 32678 pending_rounds.cc:68] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626610178732032 op_type: NO_OP noop_request { }
I20260809 14:29:54.264731 32678 raft_consensus.cc:2866] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:54.267796 32678 raft_consensus.cc:2258] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:54.282858 32655 raft_consensus.cc:504] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.286176 32514 raft_consensus.cc:2379] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3eb0bc287b12421d89abf97ca66e4d9b in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
W20260809 14:29:54.282900 32692 consensus_queue.cc:1159] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260809 14:29:54.289723 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "446c935f59874c85b29700511b23da46" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
W20260809 14:29:54.290347 32685 log.cc:924] Time spent T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Append to log took a long time: real 0.122s user 0.002s sys 0.000s
I20260809 14:29:54.290588 32333 raft_consensus.cc:3037] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:54.292043 32678 ts_tablet_manager.cc:1905] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:29:54.292300 32685 consensus_queue.cc:1159] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260809 14:29:54.288162 32655 leader_election.cc:288] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:54.295782 32677 ts_tablet_manager.cc:1905] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:54.321539 32333 raft_consensus.cc:2454] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:54.329850 31948 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050
I20260809 14:29:54.330161 32678 ts_tablet_manager.cc:1918] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:29:54.330837 32443 leader_election.cc:302] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:54.335961 32678 log.cc:1196] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/a0cb9f368ba64220b1bece7624b773b7
I20260809 14:29:54.338527 32678 ts_tablet_manager.cc:1939] T a0cb9f368ba64220b1bece7624b773b7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:54.341992 32655 raft_consensus.cc:2781] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:54.354156 32691 tablet_bootstrap.cc:492] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:54.354074 32655 raft_consensus.cc:686] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:29:54.354749 32677 ts_tablet_manager.cc:1918] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:29:54.354924 32691 ts_tablet_manager.cc:1397] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.193s user 0.010s sys 0.037s
I20260809 14:29:54.355373 32677 log.cc:1196] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/a0cb9f368ba64220b1bece7624b773b7
I20260809 14:29:54.356787 32677 ts_tablet_manager.cc:1939] T a0cb9f368ba64220b1bece7624b773b7 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:54.355808 32655 consensus_queue.cc:237] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.358430 32691 raft_consensus.cc:348] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.359546 32693 tablet_bootstrap.cc:492] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:54.359540 32691 raft_consensus.cc:374] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:54.359957 32693 ts_tablet_manager.cc:1397] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.131s user 0.024s sys 0.036s
I20260809 14:29:54.359922 32691 raft_consensus.cc:729] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:54.360917 32691 consensus_queue.cc:260] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.379596 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 446c935f59874c85b29700511b23da46
I20260809 14:29:54.391816 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 446c935f59874c85b29700511b23da46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:54 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:54.392616 32678 tablet_replica.cc:330] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:54.392673 32691 ts_tablet_manager.cc:1428] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.037s user 0.005s sys 0.002s
I20260809 14:29:54.396528 32699 tablet.cc:1722] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:29:54.402204 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "446c935f59874c85b29700511b23da46" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:54.403276 32170 raft_consensus.cc:3037] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:54.405123 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "446c935f59874c85b29700511b23da46" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:54.410144 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 446c935f59874c85b29700511b23da46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:54 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:54.413604 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_874dd4f6d8cb4d9d9d12696b12805f0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:54.415920 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_874dd4f6d8cb4d9d9d12696b12805f0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:54.430480 31938 master_service.cc:946] ReplaceTablet: received request to replace tablet 4759d2711e52406ea0ea262e395d686f from {username='slave'} at 127.0.0.1:49782
W20260809 14:29:54.433748 32095 consensus_peers.cc:487] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb -> Peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Couldn't send request to peer 6bed14aff53a41b2bbf5a13ce55dec5a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0cb9f368ba64220b1bece7624b773b7. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:29:54.435194 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 446c935f59874c85b29700511b23da46 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:54.440428 32260 leader_election.cc:302] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:54.441488 32656 raft_consensus.cc:2726] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:54.440923 32693 raft_consensus.cc:348] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.441982 32693 raft_consensus.cc:374] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:54.442345 32693 raft_consensus.cc:729] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:54.444000 32309 tablet_service.cc:1514] Processing DeleteTablet for tablet 446c935f59874c85b29700511b23da46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:54 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:29:54.444257 32099 consensus_peers.cc:487] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb -> Peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Couldn't send request to peer 3eb0bc287b12421d89abf97ca66e4d9b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0cb9f368ba64220b1bece7624b773b7. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:29:54.444877 32677 tablet_replica.cc:330] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:54.446213 32677 raft_consensus.cc:2229] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:54.446671 32677 raft_consensus.cc:2258] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:54.448482 32677 ts_tablet_manager.cc:1905] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:54.450994 32170 raft_consensus.cc:2454] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:54.451741 32679 mvcc.cc:205] Tried to move back new op lower bound from 7316626611915788288 to 7316626610178732032. Current Snapshot: MvccSnapshot[applied={T|T < 7316626611915788288}]
W20260809 14:29:54.456516 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e605554fc4174a44acf37672f4a0fd76 has no consensus state
I20260809 14:29:54.460734 32678 raft_consensus.cc:2229] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:54.462368 32678 pending_rounds.cc:62] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Trying to abort 1 pending ops.
I20260809 14:29:54.462687 32678 pending_rounds.cc:68] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626613708226560 op_type: NO_OP noop_request { }
I20260809 14:29:54.463126 32678 raft_consensus.cc:2866] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:54.463517 32678 raft_consensus.cc:2258] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:29:54.463621 32697 consensus_queue.cc:1159] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
W20260809 14:29:54.475657 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0026520a56614752b3811020593d69f9 has no consensus state
I20260809 14:29:54.476079 32678 ts_tablet_manager.cc:1905] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:54.492125 32676 ts_tablet_manager.cc:1918] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:54.493011 32676 log.cc:1196] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/b17b6faccdd74299b6c397d7f41689e6
I20260809 14:29:54.494345 32676 ts_tablet_manager.cc:1939] T b17b6faccdd74299b6c397d7f41689e6 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:54.498807 32693 consensus_queue.cc:260] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.502964 32693 ts_tablet_manager.cc:1428] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.143s user 0.007s sys 0.000s
I20260809 14:29:54.514953 32677 ts_tablet_manager.cc:1918] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:54.515364 32677 log.cc:1196] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/446c935f59874c85b29700511b23da46
I20260809 14:29:54.516018 32677 ts_tablet_manager.cc:1939] T 446c935f59874c85b29700511b23da46 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:54.522437 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_650630753b48440a94cce04fb165752e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:54.524899 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_650630753b48440a94cce04fb165752e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:54.538489 31938 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_09f478730b6647b8aad8c1632eb94910" } modify_external_catalogs: true
I20260809 14:29:54.539604 31938 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_09f478730b6647b8aad8c1632eb94910" } modify_external_catalogs: true
I20260809 14:29:54.567857 32705 tablet_bootstrap.cc:492] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:54.593114 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_982e1611f4824f7dacffbd8faefd41cf" } modify_external_catalogs: true
I20260809 14:29:54.594339 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_982e1611f4824f7dacffbd8faefd41cf" } modify_external_catalogs: true
I20260809 14:29:54.606256 32676 tablet_replica.cc:330] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:54.606060 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 446c935f59874c85b29700511b23da46 (table default.Table_9421cfe39abc4d408a26df8838273d64 [id=a53a4159cb0148e0b87cef9a21aae33c]) successfully deleted
I20260809 14:29:54.607918 32676 raft_consensus.cc:2229] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:54.609441 32678 ts_tablet_manager.cc:1918] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:54.610139 32678 log.cc:1196] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/446c935f59874c85b29700511b23da46
I20260809 14:29:54.615216 32676 pending_rounds.cc:62] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:29:54.615674 32676 pending_rounds.cc:68] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626611068080128 op_type: NO_OP noop_request { }
I20260809 14:29:54.616640 32676 raft_consensus.cc:2866] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:54.612679 32678 ts_tablet_manager.cc:1939] T 446c935f59874c85b29700511b23da46 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:54.619226 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet b17b6faccdd74299b6c397d7f41689e6 (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
I20260809 14:29:54.631910 32676 raft_consensus.cc:2258] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:54.631268 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 446c935f59874c85b29700511b23da46 (table default.Table_9421cfe39abc4d408a26df8838273d64 [id=a53a4159cb0148e0b87cef9a21aae33c]) successfully deleted
I20260809 14:29:54.643079 32676 ts_tablet_manager.cc:1905] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:54.644955 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet ca2383fa88b441e4bb31bea3aac43f2a from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:54.651994 32705 tablet_bootstrap.cc:654] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:54.670750 32676 ts_tablet_manager.cc:1918] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:54.671151 32676 log.cc:1196] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/4759d2711e52406ea0ea262e395d686f
I20260809 14:29:54.674204 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_874dd4f6d8cb4d9d9d12696b12805f0f" } new_table_name: "default.Table_844fb6abb5464617aa15aeedc254ae5c" modify_external_catalogs: true
I20260809 14:29:54.675266 31940 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet dcdd5a8f7df64614a3e8b605644f0f9a
I20260809 14:29:54.676319 32705 tablet_bootstrap.cc:492] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:54.676757 32705 ts_tablet_manager.cc:1397] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.145s user 0.005s sys 0.019s
I20260809 14:29:54.679554 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 0026520a56614752b3811020593d69f9 (DEFAULT_TABLE table=default.Table_09f478730b6647b8aad8c1632eb94910 [id=5986579051be41dd8185d74ec20a6b35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.679615 32705 raft_consensus.cc:348] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.680421 32705 raft_consensus.cc:374] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:54.680709 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0026520a56614752b3811020593d69f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.680732 32705 raft_consensus.cc:729] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:54.681442 32705 consensus_queue.cc:260] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.686383 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 0026520a56614752b3811020593d69f9 (DEFAULT_TABLE table=default.Table_09f478730b6647b8aad8c1632eb94910 [id=5986579051be41dd8185d74ec20a6b35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.687513 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0026520a56614752b3811020593d69f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.688201 32487 tablet_service.cc:1467] Processing CreateTablet for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 (DEFAULT_TABLE table=default.Table_5407d24d582f49c59ad48b0e2d5d9a4b [id=f54dffd569df4cd8a21205544a7a12b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.688192 32148 tablet_service.cc:1467] Processing CreateTablet for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 (DEFAULT_TABLE table=default.Table_5407d24d582f49c59ad48b0e2d5d9a4b [id=f54dffd569df4cd8a21205544a7a12b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.689404 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.689877 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.689690 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 0026520a56614752b3811020593d69f9 (DEFAULT_TABLE table=default.Table_09f478730b6647b8aad8c1632eb94910 [id=5986579051be41dd8185d74ec20a6b35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.690704 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0026520a56614752b3811020593d69f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.692710 32489 tablet_service.cc:1514] Processing DeleteTablet for tablet 446c935f59874c85b29700511b23da46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:54 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:54.686383 32307 tablet_service.cc:1467] Processing CreateTablet for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 (DEFAULT_TABLE table=default.Table_5407d24d582f49c59ad48b0e2d5d9a4b [id=f54dffd569df4cd8a21205544a7a12b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.693693 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.695415 32705 ts_tablet_manager.cc:1428] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.018s user 0.003s sys 0.004s
I20260809 14:29:54.700908 31939 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_1a5aedd761b04e78a73395ab7562c757" } new_table_name: "default.Table_9775d9ed12b749c49408ba829a7d5515" modify_external_catalogs: true
I20260809 14:29:54.701267 32308 tablet_service.cc:1467] Processing CreateTablet for tablet e605554fc4174a44acf37672f4a0fd76 (DEFAULT_TABLE table=default.Table_1a5aedd761b04e78a73395ab7562c757 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.702380 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e605554fc4174a44acf37672f4a0fd76. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:54.710731 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 446c935f59874c85b29700511b23da46 because the tablet was not found. No further retry: Not found: Tablet not found: 446c935f59874c85b29700511b23da46
I20260809 14:29:54.712759 32489 tablet_service.cc:1467] Processing CreateTablet for tablet e605554fc4174a44acf37672f4a0fd76 (DEFAULT_TABLE table=default.Table_1a5aedd761b04e78a73395ab7562c757 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.713847 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e605554fc4174a44acf37672f4a0fd76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.731171 32691 tablet_bootstrap.cc:492] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:54.732350 32147 tablet_service.cc:1467] Processing CreateTablet for tablet e605554fc4174a44acf37672f4a0fd76 (DEFAULT_TABLE table=default.Table_1a5aedd761b04e78a73395ab7562c757 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:54.733680 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e605554fc4174a44acf37672f4a0fd76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:54.742316 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3d42eec9be6d469dbdede7cd0553e019"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:54.743688 32691 tablet_bootstrap.cc:654] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:29:54.744648 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d42eec9be6d469dbdede7cd0553e019 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:29:54.747207 32671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 14:29:54.748246 32671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 14:29:54.749308 32671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 14:29:54.671957 32676 ts_tablet_manager.cc:1939] T 4759d2711e52406ea0ea262e395d686f P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:54.753139 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_a8d4268684564e2e988b1e2bc08f779c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:54.755646 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8d4268684564e2e988b1e2bc08f779c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:54.756386 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 4759d2711e52406ea0ea262e395d686f (table default.Table_f70c6ac8bdfc4aadb7fbb37cf148ac58 [id=5504fd425a0b4fb4bcd5acb5d506475f]) successfully deleted
I20260809 14:29:54.770462 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_aceab181a5d344c78855f1c07a63e553"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:54.772146 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aceab181a5d344c78855f1c07a63e553 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:54.776230 32691 tablet_bootstrap.cc:492] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:54.776851 32691 ts_tablet_manager.cc:1397] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.046s user 0.015s sys 0.000s
I20260809 14:29:54.780807 32705 tablet_bootstrap.cc:492] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:54.784228 32676 tablet_replica.cc:330] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:54.791450 32691 raft_consensus.cc:348] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.792330 32691 raft_consensus.cc:374] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:54.792781 32691 raft_consensus.cc:729] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:54.793589 32691 consensus_queue.cc:260] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.793143 32676 raft_consensus.cc:2229] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:54.799713 32691 ts_tablet_manager.cc:1428] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.022s user 0.005s sys 0.003s
I20260809 14:29:54.804499 32693 tablet_bootstrap.cc:492] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:54.815719 32693 tablet_bootstrap.cc:654] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:54.820294 32691 tablet_bootstrap.cc:492] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:54.821501 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet ca2383fa88b441e4bb31bea3aac43f2a
W20260809 14:29:54.821637 32671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 14:29:54.822000 32671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 14:29:54.822398 32671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 14:29:54.822726 32671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 14:29:54.823194 31942 catalog_manager.cc:6565] ReplaceTablet: tablet ca2383fa88b441e4bb31bea3aac43f2a deleted and replaced by tablet 84c8eb8ad2dd406085779ed6a86eb70e
I20260809 14:29:54.826253 32309 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2383fa88b441e4bb31bea3aac43f2a with delete_type TABLET_DATA_DELETED (replaced by tablet 84c8eb8ad2dd406085779ed6a86eb70e) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:54.790462 32705 tablet_bootstrap.cc:654] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:54.827078 32677 tablet_replica.cc:330] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:54.827721 32677 raft_consensus.cc:2229] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:54.827733 32676 raft_consensus.cc:2258] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:54.828163 32677 raft_consensus.cc:2258] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:54.833806 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2383fa88b441e4bb31bea3aac43f2a with delete_type TABLET_DATA_DELETED (replaced by tablet 84c8eb8ad2dd406085779ed6a86eb70e) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:54.834714 32678 tablet_replica.cc:330] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:54.835654 32678 raft_consensus.cc:2229] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:54.835871 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2383fa88b441e4bb31bea3aac43f2a with delete_type TABLET_DATA_DELETED (replaced by tablet 84c8eb8ad2dd406085779ed6a86eb70e) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:54.836102 32678 raft_consensus.cc:2258] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:54.836822 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e1f46b3673cb4694925d4248e8ef6c81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:54.839715 32678 ts_tablet_manager.cc:1905] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:54.839629 32676 ts_tablet_manager.cc:1905] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:29:54.843086 31939 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e605554fc4174a44acf37672f4a0fd76 consensus state has no leader
I20260809 14:29:54.848138 32667 raft_consensus.cc:482] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:54.848824 32667 raft_consensus.cc:504] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.852363 32677 ts_tablet_manager.cc:1905] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:54.854640 32705 tablet_bootstrap.cc:492] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:54.854269 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca2383fa88b441e4bb31bea3aac43f2a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:54.855085 32705 ts_tablet_manager.cc:1397] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.075s user 0.013s sys 0.007s
I20260809 14:29:54.854866 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca2383fa88b441e4bb31bea3aac43f2a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:29:54.857030 32099 leader_election.cc:341] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: Tablet not RUNNING: STOPPED
W20260809 14:29:54.859323 32095 leader_election.cc:341] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Illegal state: Tablet not RUNNING: STOPPED
I20260809 14:29:54.860138 32095 leader_election.cc:302] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:29:54.862383 32701 raft_consensus.cc:2726] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:54.863579 32667 leader_election.cc:288] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:29:54.864706 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1f46b3673cb4694925d4248e8ef6c81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:54.860383 32705 raft_consensus.cc:348] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:54.873247 32705 raft_consensus.cc:374] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:54.873582 32705 raft_consensus.cc:729] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:54.873769 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:54.875001 32146 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2383fa88b441e4bb31bea3aac43f2a with delete_type TABLET_DATA_DELETED (replaced by tablet 84c8eb8ad2dd406085779ed6a86eb70e) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:54.879119 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_11c018a7f05149e18f070bf70b2f1bcc" } modify_external_catalogs: true
I20260809 14:29:54.875016 32705 consensus_queue.cc:260] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
W20260809 14:29:54.885118 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5951ffedd0cd481f9f425f0af3b8ad01 has no consensus state
I20260809 14:29:54.885169 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_11c018a7f05149e18f070bf70b2f1bcc" } modify_external_catalogs: true
I20260809 14:29:54.887034 32705 ts_tablet_manager.cc:1428] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.032s user 0.010s sys 0.002s
I20260809 14:29:54.895237 32691 tablet_bootstrap.cc:654] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:54.901425 31939 master_service.cc:946] ReplaceTablet: received request to replace tablet a0cb9f368ba64220b1bece7624b773b7 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:54.906258 32677 ts_tablet_manager.cc:1918] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:54.906647 32677 log.cc:1196] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/ca2383fa88b441e4bb31bea3aac43f2a
I20260809 14:29:54.907508 32677 ts_tablet_manager.cc:1939] T ca2383fa88b441e4bb31bea3aac43f2a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:54.912581 32705 tablet_bootstrap.cc:492] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:54.920684 32691 tablet_bootstrap.cc:492] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:54.921123 32691 ts_tablet_manager.cc:1397] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.101s user 0.010s sys 0.003s
I20260809 14:29:54.921702 32705 tablet_bootstrap.cc:654] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:54.923660 32691 raft_consensus.cc:348] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:54.924959 32676 ts_tablet_manager.cc:1918] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:29:54.925486 32676 log.cc:1196] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/a0cb9f368ba64220b1bece7624b773b7
I20260809 14:29:54.926630 32676 ts_tablet_manager.cc:1939] T a0cb9f368ba64220b1bece7624b773b7 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:54.927919 32678 ts_tablet_manager.cc:1918] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:54.930953 32676 tablet_replica.cc:330] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:54.933897 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet ca2383fa88b441e4bb31bea3aac43f2a (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
I20260809 14:29:54.934789 32676 raft_consensus.cc:2229] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:54.937809 32705 tablet_bootstrap.cc:492] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:54.938498 32693 tablet_bootstrap.cc:492] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:54.938408 32705 ts_tablet_manager.cc:1397] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.002s
I20260809 14:29:54.938994 32693 ts_tablet_manager.cc:1397] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.135s user 0.011s sys 0.013s
I20260809 14:29:54.942129 32705 raft_consensus.cc:348] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:54.942397 32693 raft_consensus.cc:348] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.942847 32705 raft_consensus.cc:374] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:54.943133 32693 raft_consensus.cc:374] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:54.943117 32705 raft_consensus.cc:729] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:54.943416 32693 raft_consensus.cc:729] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:54.928292 32678 log.cc:1196] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/ca2383fa88b441e4bb31bea3aac43f2a
I20260809 14:29:54.943800 32676 raft_consensus.cc:2258] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:54.944550 32678 ts_tablet_manager.cc:1939] T ca2383fa88b441e4bb31bea3aac43f2a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:54.946884 32676 ts_tablet_manager.cc:1905] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:54.948417 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet ca2383fa88b441e4bb31bea3aac43f2a (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
I20260809 14:29:54.924371 32691 raft_consensus.cc:374] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:54.957614 32691 raft_consensus.cc:729] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:54.958551 32691 consensus_queue.cc:260] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
W20260809 14:29:54.967048 31933 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5951ffedd0cd481f9f425f0af3b8ad01 (table default.Table_844fb6abb5464617aa15aeedc254ae5c [id=104d9a5015094e82a8cac7ea3b45c0fc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5951ffedd0cd481f9f425f0af3b8ad01 has no consensus state
I20260809 14:29:54.970628 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet e605554fc4174a44acf37672f4a0fd76 (table default.Table_9775d9ed12b749c49408ba829a7d5515 [id=032f6264a2c842f1ba6b007331ccfcc6]). Expected version 2 got 0
I20260809 14:29:54.973558 32705 consensus_queue.cc:260] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:54.975876 32691 ts_tablet_manager.cc:1428] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.054s user 0.006s sys 0.000s
I20260809 14:29:54.976120 32705 ts_tablet_manager.cc:1428] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.037s user 0.008s sys 0.001s
I20260809 14:29:54.976564 32693 consensus_queue.cc:260] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:54.977310 32705 tablet_bootstrap.cc:492] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:54.977936 32678 tablet_replica.cc:330] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:54.998473 32656 raft_consensus.cc:482] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:54.999208 32656 raft_consensus.cc:504] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:55.002414 32656 leader_election.cc:288] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:55.010341 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98e5d4c2bd54d3f9fa343d9d1dc4050" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:55.011039 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98e5d4c2bd54d3f9fa343d9d1dc4050" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:55.011394 32514 raft_consensus.cc:2454] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
W20260809 14:29:55.013707 32260 leader_election.cc:341] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:55.020913 32705 tablet_bootstrap.cc:654] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.021270 32260 leader_election.cc:302] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:55.021379 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:55.022625 32656 raft_consensus.cc:2781] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:55.023036 32656 raft_consensus.cc:482] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:55.023407 32656 raft_consensus.cc:3037] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:29:55.023995 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 already in progress: creating tablet
I20260809 14:29:55.026762 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet e605554fc4174a44acf37672f4a0fd76 (table default.Table_9775d9ed12b749c49408ba829a7d5515 [id=032f6264a2c842f1ba6b007331ccfcc6]). Expected version 2 got 0
I20260809 14:29:55.035888 31939 master_service.cc:946] ReplaceTablet: received request to replace tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:55.037331 32691 tablet_bootstrap.cc:492] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:55.041254 32693 ts_tablet_manager.cc:1428] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.102s user 0.008s sys 0.000s
I20260809 14:29:55.042090 32678 raft_consensus.cc:2229] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.044384 32678 raft_consensus.cc:2258] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.045717 32676 ts_tablet_manager.cc:1918] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:55.046828 32676 log.cc:1196] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/446c935f59874c85b29700511b23da46
I20260809 14:29:55.047501 32676 ts_tablet_manager.cc:1939] T 446c935f59874c85b29700511b23da46 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:55.050736 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 446c935f59874c85b29700511b23da46 (table default.Table_9421cfe39abc4d408a26df8838273d64 [id=a53a4159cb0148e0b87cef9a21aae33c]) successfully deleted
I20260809 14:29:55.056532 32676 tablet_replica.cc:330] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:55.057046 32671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.252.84:46315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.30.252.126:43223,127.30.252.125:42169,127.30.252.124:34903
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.252.126:43223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.252.126
--webserver_port=39845
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 13:36:37 UTC on ccycloud.quasar-iqymvz.root.comops.site
build id 4576526
TSAN enabled
I20260809 14:29:55.057209 32676 raft_consensus.cc:2229] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.058504 32676 raft_consensus.cc:2258] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.049156 32693 tablet_bootstrap.cc:492] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:55.058809 32671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 14:29:55.062187 32671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 14:29:55.061364 32678 ts_tablet_manager.cc:1905] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.064297 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_844fb6abb5464617aa15aeedc254ae5c" } new_table_name: "default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f" modify_external_catalogs: true
I20260809 14:29:55.066069 32676 ts_tablet_manager.cc:1905] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.082126 32691 tablet_bootstrap.cc:654] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.087533 32693 tablet_bootstrap.cc:654] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.098555 32705 tablet_bootstrap.cc:492] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:55.099138 32705 ts_tablet_manager.cc:1397] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.122s user 0.013s sys 0.021s
I20260809 14:29:55.102831 32676 ts_tablet_manager.cc:1918] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:55.103282 32676 log.cc:1196] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/ca2383fa88b441e4bb31bea3aac43f2a
I20260809 14:29:55.104007 32676 ts_tablet_manager.cc:1939] T ca2383fa88b441e4bb31bea3aac43f2a P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:55.103660 32705 raft_consensus.cc:348] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:55.104991 32705 raft_consensus.cc:374] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.105290 32705 raft_consensus.cc:729] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:55.105773 31942 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet deb083d7f17c48e8b0c01d12205411c7
I20260809 14:29:55.105969 32705 consensus_queue.cc:260] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:55.107941 31937 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9775d9ed12b749c49408ba829a7d5515" } new_table_name: "default.Table_5680c3dd85d14f089c9ef93afaf56372" modify_external_catalogs: true
I20260809 14:29:55.109936 32705 ts_tablet_manager.cc:1428] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20260809 14:29:55.089958 32656 raft_consensus.cc:504] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:55.126780 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98e5d4c2bd54d3f9fa343d9d1dc4050" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:55.127581 32170 raft_consensus.cc:3037] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:29:55.130656 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5951ffedd0cd481f9f425f0af3b8ad01 has no consensus state
I20260809 14:29:55.136950 31943 master_service.cc:946] ReplaceTablet: received request to replace tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:55.137573 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet ca2383fa88b441e4bb31bea3aac43f2a (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
I20260809 14:29:55.139904 32656 leader_election.cc:288] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:55.140179 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98e5d4c2bd54d3f9fa343d9d1dc4050" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
W20260809 14:29:55.141808 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet ca2383fa88b441e4bb31bea3aac43f2a because the tablet was not found. No further retry: Not found: Tablet not found: ca2383fa88b441e4bb31bea3aac43f2a
W20260809 14:29:55.142015 32260 leader_election.cc:341] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Illegal state: Tablet not RUNNING: STOPPED
I20260809 14:29:55.142338 32170 raft_consensus.cc:2454] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:29:55.143561 32260 leader_election.cc:302] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:29:55.144742 32656 raft_consensus.cc:2781] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:55.153450 31939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet e98e5d4c2bd54d3f9fa343d9d1dc4050
I20260809 14:29:55.154999 31939 catalog_manager.cc:6565] ReplaceTablet: tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 deleted and replaced by tablet 0853cf76a42d4db1a2ef1f698371c88f
I20260809 14:29:55.155274 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 with delete_type TABLET_DATA_DELETED (replaced by tablet 0853cf76a42d4db1a2ef1f698371c88f) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:55.156044 32676 tablet_replica.cc:330] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:55.156030 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 with delete_type TABLET_DATA_DELETED (replaced by tablet 0853cf76a42d4db1a2ef1f698371c88f) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:55.157213 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 with delete_type TABLET_DATA_DELETED (replaced by tablet 0853cf76a42d4db1a2ef1f698371c88f) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:55.157213 32676 raft_consensus.cc:2229] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.157845 32676 raft_consensus.cc:2258] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:29:55.159185 32724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:29:55.159487 32677 tablet_replica.cc:330] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:55.164707 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_dadc1d59ab5f4b799d8d5d1cf23a3580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:55.166608 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dadc1d59ab5f4b799d8d5d1cf23a3580 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.171065 32656 raft_consensus.cc:686] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:29:55.172202 32656 consensus_queue.cc:237] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:55.172860 32676 ts_tablet_manager.cc:1905] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:29:55.177276 31937 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e605554fc4174a44acf37672f4a0fd76 consensus state has no leader
I20260809 14:29:55.189702 32691 tablet_bootstrap.cc:492] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:55.190255 32691 ts_tablet_manager.cc:1397] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.153s user 0.010s sys 0.003s
I20260809 14:29:55.199635 32693 tablet_bootstrap.cc:492] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:55.200169 32693 ts_tablet_manager.cc:1397] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.151s user 0.012s sys 0.000s
I20260809 14:29:55.200923 32691 raft_consensus.cc:348] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.207811 32667 raft_consensus.cc:482] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:55.208400 32667 raft_consensus.cc:504] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.209796 32691 raft_consensus.cc:374] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.210988 32691 raft_consensus.cc:729] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:55.212757 32691 consensus_queue.cc:260] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.214156 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0026520a56614752b3811020593d69f9" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:55.215677 32667 leader_election.cc:288] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:55.217654 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_733ab1fa87b647d2961d60a43900c952"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:55.219818 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_733ab1fa87b647d2961d60a43900c952 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.203987 32693 raft_consensus.cc:348] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.215101 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0026520a56614752b3811020593d69f9" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:55.225520 32514 raft_consensus.cc:2454] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:55.227178 31938 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 0026520a56614752b3811020593d69f9
I20260809 14:29:55.227227 32095 leader_election.cc:302] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:55.228510 32667 raft_consensus.cc:2781] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:55.228922 32667 raft_consensus.cc:482] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:55.229342 32667 raft_consensus.cc:3037] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:55.230540 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 0026520a56614752b3811020593d69f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:54 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:55.235271 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 0026520a56614752b3811020593d69f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:54 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:55.237476 32491 tablet_service.cc:1467] Processing CreateTablet for tablet c48a1586a9164629a7dfe057bbf12dc0 (DEFAULT_TABLE table=default.Table_e768d5aab10146f3be037da4134ae39d [id=6fac5819a44f4152bfe254bf83ef692d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.237715 32701 raft_consensus.cc:482] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:55.238708 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48a1586a9164629a7dfe057bbf12dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.238267 32701 raft_consensus.cc:504] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.246093 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e605554fc4174a44acf37672f4a0fd76" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:55.246676 32514 raft_consensus.cc:2454] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:55.247009 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_7d8d4891be3a46e8adc1a2074020612c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:55.247866 32095 leader_election.cc:302] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
W20260809 14:29:55.248426 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d8d4891be3a46e8adc1a2074020612c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.240273 32701 leader_election.cc:288] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:55.251605 32701 raft_consensus.cc:2781] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:55.251890 32701 raft_consensus.cc:482] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:55.252166 32701 raft_consensus.cc:3037] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:55.254119 32148 tablet_service.cc:1467] Processing CreateTablet for tablet c48a1586a9164629a7dfe057bbf12dc0 (DEFAULT_TABLE table=default.Table_e768d5aab10146f3be037da4134ae39d [id=6fac5819a44f4152bfe254bf83ef692d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.255543 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48a1586a9164629a7dfe057bbf12dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.257946 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 664f49ed04144a59a8fd6dd1e61a6907 (DEFAULT_TABLE table=default.Table_8580644785dc4eaab7fefc3d44b66089 [id=a50d475ae87f4c1ebe36017da6ac772e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.259006 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 664f49ed04144a59a8fd6dd1e61a6907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.264659 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 664f49ed04144a59a8fd6dd1e61a6907 (DEFAULT_TABLE table=default.Table_8580644785dc4eaab7fefc3d44b66089 [id=a50d475ae87f4c1ebe36017da6ac772e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.265755 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 664f49ed04144a59a8fd6dd1e61a6907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.276978 32667 raft_consensus.cc:504] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.234524 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 0026520a56614752b3811020593d69f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:54 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:55.242532 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e605554fc4174a44acf37672f4a0fd76" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:55.244125 32309 tablet_service.cc:1467] Processing CreateTablet for tablet c48a1586a9164629a7dfe057bbf12dc0 (DEFAULT_TABLE table=default.Table_e768d5aab10146f3be037da4134ae39d [id=6fac5819a44f4152bfe254bf83ef692d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.255194 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 664f49ed04144a59a8fd6dd1e61a6907 (DEFAULT_TABLE table=default.Table_8580644785dc4eaab7fefc3d44b66089 [id=a50d475ae87f4c1ebe36017da6ac772e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.289006 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 664f49ed04144a59a8fd6dd1e61a6907. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:55.289002 32099 leader_election.cc:341] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:55.290449 32701 raft_consensus.cc:504] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.291162 31938 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet e605554fc4174a44acf37672f4a0fd76 (table default.Table_5680c3dd85d14f089c9ef93afaf56372 [id=032f6264a2c842f1ba6b007331ccfcc6]). Expected version 3 got 0
I20260809 14:29:55.263897 32677 raft_consensus.cc:2229] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:55.293498 32677 pending_rounds.cc:62] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 1 pending ops.
I20260809 14:29:55.293741 32677 pending_rounds.cc:68] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626617123188736 op_type: NO_OP noop_request { }
I20260809 14:29:55.294180 32677 raft_consensus.cc:2866] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:55.294673 32677 raft_consensus.cc:2258] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.284582 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 5951ffedd0cd481f9f425f0af3b8ad01 (DEFAULT_TABLE table=default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.295738 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5951ffedd0cd481f9f425f0af3b8ad01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.295859 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0026520a56614752b3811020593d69f9" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:55.298979 32678 ts_tablet_manager.cc:1918] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:55.299463 32678 log.cc:1196] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/e98e5d4c2bd54d3f9fa343d9d1dc4050
I20260809 14:29:55.300235 32678 ts_tablet_manager.cc:1939] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:55.304143 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48a1586a9164629a7dfe057bbf12dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.304279 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e605554fc4174a44acf37672f4a0fd76" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
W20260809 14:29:55.305958 32099 leader_election.cc:341] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:55.300351 32667 leader_election.cc:288] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:55.308177 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 5951ffedd0cd481f9f425f0af3b8ad01 (DEFAULT_TABLE table=default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:29:55.308539 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 60748.7us
I20260809 14:29:55.309355 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5951ffedd0cd481f9f425f0af3b8ad01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.308995 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 765d102c8c6444b89da084f27a4cf8ed (DEFAULT_TABLE table=default.Table_650630753b48440a94cce04fb165752e [id=5cfe6abc061c4d39a8da29bc24541eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.312773 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 765d102c8c6444b89da084f27a4cf8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.316880 32701 leader_election.cc:288] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:55.310565 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0026520a56614752b3811020593d69f9" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:55.319386 32514 raft_consensus.cc:3037] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:55.319486 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 765d102c8c6444b89da084f27a4cf8ed (DEFAULT_TABLE table=default.Table_650630753b48440a94cce04fb165752e [id=5cfe6abc061c4d39a8da29bc24541eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.318349 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e605554fc4174a44acf37672f4a0fd76" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:55.320727 32515 raft_consensus.cc:3037] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:55.324646 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 765d102c8c6444b89da084f27a4cf8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.325114 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 (table default.Table_5407d24d582f49c59ad48b0e2d5d9a4b [id=f54dffd569df4cd8a21205544a7a12b7]) successfully deleted
W20260809 14:29:55.325866 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 because the tablet was not found. No further retry: Not found: Tablet not found: e98e5d4c2bd54d3f9fa343d9d1dc4050
W20260809 14:29:55.326498 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 0026520a56614752b3811020593d69f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0026520a56614752b3811020593d69f9 already in progress: creating tablet
I20260809 14:29:55.327616 32514 raft_consensus.cc:2454] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:29:55.328969 32095 leader_election.cc:302] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:29:55.329356 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 765d102c8c6444b89da084f27a4cf8ed (DEFAULT_TABLE table=default.Table_650630753b48440a94cce04fb165752e [id=5cfe6abc061c4d39a8da29bc24541eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.330413 32701 raft_consensus.cc:2781] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:55.330605 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 765d102c8c6444b89da084f27a4cf8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.330897 32701 raft_consensus.cc:686] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:29:55.335630 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e768d5aab10146f3be037da4134ae39d" } new_table_name: "default.Table_440a4f8d04c24bf682668ec6d4d12a4d" modify_external_catalogs: true
I20260809 14:29:55.342240 32677 ts_tablet_manager.cc:1905] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.341830 32701 consensus_queue.cc:237] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.343703 32676 ts_tablet_manager.cc:1918] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:55.344678 32676 log.cc:1196] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/e98e5d4c2bd54d3f9fa343d9d1dc4050
I20260809 14:29:55.345165 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8580644785dc4eaab7fefc3d44b66089" } modify_external_catalogs: true
I20260809 14:29:55.345789 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8580644785dc4eaab7fefc3d44b66089" } modify_external_catalogs: true
I20260809 14:29:55.346484 32676 ts_tablet_manager.cc:1939] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:55.347549 32515 raft_consensus.cc:2454] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:29:55.287277 32693 raft_consensus.cc:374] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.348946 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_6c2d09c5ada541da918808352f079b25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:55.350806 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c2d09c5ada541da918808352f079b25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.286527 32691 ts_tablet_manager.cc:1428] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.096s user 0.003s sys 0.003s
I20260809 14:29:55.357878 32691 tablet_bootstrap.cc:492] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:55.348344 32693 raft_consensus.cc:729] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:55.365851 32693 consensus_queue.cc:260] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.366782 32333 raft_consensus.cc:2454] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:55.372773 32095 leader_election.cc:302] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:29:55.374368 32731 raft_consensus.cc:2781] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:55.375027 32731 raft_consensus.cc:686] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:29:55.375924 32731 consensus_queue.cc:237] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.378434 32676 tablet_replica.cc:330] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:55.381901 32693 ts_tablet_manager.cc:1428] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.181s user 0.006s sys 0.000s
I20260809 14:29:55.382665 32676 raft_consensus.cc:2229] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:55.383379 32676 pending_rounds.cc:62] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:29:55.386950 32693 tablet_bootstrap.cc:492] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:55.352669 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 (table default.Table_5407d24d582f49c59ad48b0e2d5d9a4b [id=f54dffd569df4cd8a21205544a7a12b7]) successfully deleted
W20260809 14:29:55.364058 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c48a1586a9164629a7dfe057bbf12dc0 consensus state has no leader
I20260809 14:29:55.373543 31937 master_service.cc:946] ReplaceTablet: received request to replace tablet e605554fc4174a44acf37672f4a0fd76 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:55.376035 31944 master_service.cc:946] ReplaceTablet: received request to replace tablet 0026520a56614752b3811020593d69f9 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:55.384078 32676 pending_rounds.cc:68] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626617767624704 op_type: NO_OP noop_request { }
I20260809 14:29:55.393654 32676 raft_consensus.cc:2866] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:55.381063 31939 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_650630753b48440a94cce04fb165752e" } new_table_name: "default.Table_9ecd2a39ee0f40cca949c529cba934ac" modify_external_catalogs: true
I20260809 14:29:55.394697 32676 raft_consensus.cc:2258] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.395839 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 5951ffedd0cd481f9f425f0af3b8ad01 (DEFAULT_TABLE table=default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.398758 32676 ts_tablet_manager.cc:1905] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.399646 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5951ffedd0cd481f9f425f0af3b8ad01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.411065 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 0026520a56614752b3811020593d69f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:54 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:55.411875 32678 tablet_replica.cc:330] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:55.412027 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_ade4d74e855d41939005da22dbe0cde9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:55.412804 32678 raft_consensus.cc:2229] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.413304 32678 raft_consensus.cc:2258] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:29:55.414115 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ade4d74e855d41939005da22dbe0cde9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.415604 32678 ts_tablet_manager.cc:1905] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.421391 32705 tablet_bootstrap.cc:492] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:55.441583 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_a8d4268684564e2e988b1e2bc08f779c" } modify_external_catalogs: true
I20260809 14:29:55.442247 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_a8d4268684564e2e988b1e2bc08f779c" } modify_external_catalogs: true
W20260809 14:29:55.467089 31939 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 765d102c8c6444b89da084f27a4cf8ed consensus state has no leader
I20260809 14:29:55.470870 32693 tablet_bootstrap.cc:654] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.475790 32677 ts_tablet_manager.cc:1918] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:55.477531 32677 log.cc:1196] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/e98e5d4c2bd54d3f9fa343d9d1dc4050
I20260809 14:29:55.478271 32677 ts_tablet_manager.cc:1939] T e98e5d4c2bd54d3f9fa343d9d1dc4050 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:55.480600 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 6e873f3b715c424e8693dbcba0990fad (DEFAULT_TABLE table=default.Table_733ab1fa87b647d2961d60a43900c952 [id=0d4ec30e1ef44803b533141ba0e57fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.482292 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet e98e5d4c2bd54d3f9fa343d9d1dc4050 (table default.Table_5407d24d582f49c59ad48b0e2d5d9a4b [id=f54dffd569df4cd8a21205544a7a12b7]) successfully deleted
I20260809 14:29:55.485033 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_68743f87e3364e37a579888759048d44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:55.485636 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 6e873f3b715c424e8693dbcba0990fad (DEFAULT_TABLE table=default.Table_733ab1fa87b647d2961d60a43900c952 [id=0d4ec30e1ef44803b533141ba0e57fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.486843 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e873f3b715c424e8693dbcba0990fad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:55.487059 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68743f87e3364e37a579888759048d44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.488373 32677 tablet_replica.cc:330] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:55.489117 32677 raft_consensus.cc:2229] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.489630 32677 raft_consensus.cc:2258] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.490443 32676 ts_tablet_manager.cc:1918] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:55.491701 32676 log.cc:1196] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/0026520a56614752b3811020593d69f9
I20260809 14:29:55.492875 32676 ts_tablet_manager.cc:1939] T 0026520a56614752b3811020593d69f9 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:55.493386 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 6e873f3b715c424e8693dbcba0990fad (DEFAULT_TABLE table=default.Table_733ab1fa87b647d2961d60a43900c952 [id=0d4ec30e1ef44803b533141ba0e57fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.494503 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e873f3b715c424e8693dbcba0990fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.494678 32691 tablet_bootstrap.cc:654] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.498545 32678 ts_tablet_manager.cc:1918] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:55.498939 32678 log.cc:1196] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/0026520a56614752b3811020593d69f9
I20260809 14:29:55.499583 32678 ts_tablet_manager.cc:1939] T 0026520a56614752b3811020593d69f9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:55.501364 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 0026520a56614752b3811020593d69f9 (table default.Table_09f478730b6647b8aad8c1632eb94910 [id=5986579051be41dd8185d74ec20a6b35]) successfully deleted
I20260809 14:29:55.501745 32677 ts_tablet_manager.cc:1905] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.502231 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e873f3b715c424e8693dbcba0990fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.503975 32693 tablet_bootstrap.cc:492] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:55.509655 32693 ts_tablet_manager.cc:1397] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.123s user 0.006s sys 0.006s
I20260809 14:29:55.513453 32693 raft_consensus.cc:348] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.514259 32693 raft_consensus.cc:374] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.513783 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_7d6f721ab1324625bcf5ac42e3e20f8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:55.514559 32693 raft_consensus.cc:729] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:29:55.516902 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d6f721ab1324625bcf5ac42e3e20f8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.520442 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 0026520a56614752b3811020593d69f9 (table default.Table_09f478730b6647b8aad8c1632eb94910 [id=5986579051be41dd8185d74ec20a6b35]) successfully deleted
I20260809 14:29:55.521530 32693 consensus_queue.cc:260] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.529875 32693 ts_tablet_manager.cc:1428] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.020s user 0.004s sys 0.003s
I20260809 14:29:55.534982 32693 tablet_bootstrap.cc:492] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:55.550367 32693 tablet_bootstrap.cc:654] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.556170 32677 ts_tablet_manager.cc:1918] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:55.556608 32677 log.cc:1196] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/0026520a56614752b3811020593d69f9
I20260809 14:29:55.557308 32677 ts_tablet_manager.cc:1939] T 0026520a56614752b3811020593d69f9 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:55.559468 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 969849d30c204937a7508e8d54dae427
I20260809 14:29:55.560149 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 0026520a56614752b3811020593d69f9 (table default.Table_09f478730b6647b8aad8c1632eb94910 [id=5986579051be41dd8185d74ec20a6b35]) successfully deleted
I20260809 14:29:55.564304 32693 tablet_bootstrap.cc:492] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:55.564301 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 969849d30c204937a7508e8d54dae427 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:55.564936 32693 ts_tablet_manager.cc:1397] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.011s
I20260809 14:29:55.565459 32705 tablet_bootstrap.cc:654] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.569098 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 969849d30c204937a7508e8d54dae427 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:44626
W20260809 14:29:55.569662 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 969849d30c204937a7508e8d54dae427 because the tablet was not found. No further retry: Not found: Tablet not found: 969849d30c204937a7508e8d54dae427
I20260809 14:29:55.568153 32693 raft_consensus.cc:348] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.570924 32693 raft_consensus.cc:374] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.571233 32693 raft_consensus.cc:729] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:29:55.571448 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 969849d30c204937a7508e8d54dae427 because the tablet was not found. No further retry: Not found: Tablet not found: 969849d30c204937a7508e8d54dae427
I20260809 14:29:55.571981 32693 consensus_queue.cc:260] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.572677 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet 969849d30c204937a7508e8d54dae427 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:55.574882 32691 tablet_bootstrap.cc:492] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
W20260809 14:29:55.578574 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 969849d30c204937a7508e8d54dae427 because the tablet was not found. No further retry: Not found: Tablet not found: 969849d30c204937a7508e8d54dae427
I20260809 14:29:55.581269 32691 ts_tablet_manager.cc:1397] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.224s user 0.006s sys 0.008s
I20260809 14:29:55.582758 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 049678aadd4549d1b831174727647ec8 (DEFAULT_TABLE table=default.Table_7d8d4891be3a46e8adc1a2074020612c [id=1240bfe259ae4a2596755c9b3d3d7474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.584206 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049678aadd4549d1b831174727647ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.585525 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 049678aadd4549d1b831174727647ec8 (DEFAULT_TABLE table=default.Table_7d8d4891be3a46e8adc1a2074020612c [id=1240bfe259ae4a2596755c9b3d3d7474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.585498 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 049678aadd4549d1b831174727647ec8 (DEFAULT_TABLE table=default.Table_7d8d4891be3a46e8adc1a2074020612c [id=1240bfe259ae4a2596755c9b3d3d7474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.585908 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 658ab8e11b37419ba2176a9175bfd936 (DEFAULT_TABLE table=default.Table_e1f46b3673cb4694925d4248e8ef6c81 [id=fa8fcb1d6ccb436080febae3ea991892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.586696 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049678aadd4549d1b831174727647ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.587112 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658ab8e11b37419ba2176a9175bfd936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.588141 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049678aadd4549d1b831174727647ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.602602 32693 ts_tablet_manager.cc:1428] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.037s user 0.007s sys 0.000s
I20260809 14:29:55.603868 32693 tablet_bootstrap.cc:492] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:55.619012 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 849d798165144531810a03dc2b9b5f1e (DEFAULT_TABLE table=default.Table_aceab181a5d344c78855f1c07a63e553 [id=45f3914d4c214f1587b0bea324c32a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.620283 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 849d798165144531810a03dc2b9b5f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.620409 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet 664f49ed04144a59a8fd6dd1e61a6907 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:55.631062 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_6c2d09c5ada541da918808352f079b25" } new_table_name: "default.Table_379bcab3d28446218fcd18205d6bcea0" modify_external_catalogs: true
I20260809 14:29:55.631788 32310 tablet_service.cc:1467] Processing CreateTablet for tablet c468382a28934d50babf7bbc9a834c08 (DEFAULT_TABLE table=default.Table_3d42eec9be6d469dbdede7cd0553e019 [id=44f0605ad9e4418ca64ceca2a4120cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.632939 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 658ab8e11b37419ba2176a9175bfd936 (DEFAULT_TABLE table=default.Table_e1f46b3673cb4694925d4248e8ef6c81 [id=fa8fcb1d6ccb436080febae3ea991892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.635195 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c468382a28934d50babf7bbc9a834c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.636341 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 658ab8e11b37419ba2176a9175bfd936 (DEFAULT_TABLE table=default.Table_e1f46b3673cb4694925d4248e8ef6c81 [id=fa8fcb1d6ccb436080febae3ea991892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.638022 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658ab8e11b37419ba2176a9175bfd936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.645762 32705 tablet_bootstrap.cc:492] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:55.629231 32691 raft_consensus.cc:348] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.646951 32705 ts_tablet_manager.cc:1397] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.226s user 0.006s sys 0.020s
I20260809 14:29:55.647547 32691 raft_consensus.cc:374] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.650398 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 849d798165144531810a03dc2b9b5f1e (DEFAULT_TABLE table=default.Table_aceab181a5d344c78855f1c07a63e553 [id=45f3914d4c214f1587b0bea324c32a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.634447 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658ab8e11b37419ba2176a9175bfd936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.651664 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3b3fded95fa74c4086529b42834a0955"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:55.652601 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 849d798165144531810a03dc2b9b5f1e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:55.653421 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b3fded95fa74c4086529b42834a0955 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.654894 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 664f49ed04144a59a8fd6dd1e61a6907
I20260809 14:29:55.648329 32691 raft_consensus.cc:729] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:55.662542 32705 raft_consensus.cc:348] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.663249 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 849d798165144531810a03dc2b9b5f1e (DEFAULT_TABLE table=default.Table_aceab181a5d344c78855f1c07a63e553 [id=45f3914d4c214f1587b0bea324c32a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.663686 32693 tablet_bootstrap.cc:654] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.664549 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 849d798165144531810a03dc2b9b5f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.667066 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 664f49ed04144a59a8fd6dd1e61a6907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:55.672398 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_aceab181a5d344c78855f1c07a63e553" } new_table_name: "default.Table_3228a1bca97b4dfbbbcb8095a1da792e" modify_external_catalogs: true
I20260809 14:29:55.672648 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet e605554fc4174a44acf37672f4a0fd76
W20260809 14:29:55.673702 31930 consensus_peers.cc:487] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 -> Peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Couldn't send request to peer 9d10512ac8b3406098fad1f7da012ab3. Status: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260809 14:29:55.673900 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 664f49ed04144a59a8fd6dd1e61a6907 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 664f49ed04144a59a8fd6dd1e61a6907 already in progress: creating tablet
I20260809 14:29:55.676529 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet e605554fc4174a44acf37672f4a0fd76 with delete_type TABLET_DATA_DELETED (replaced by tablet 71aad93fcc8b42e6b55db6b3ed6b4f91) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:55.677434 32678 tablet_replica.cc:330] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:55.678071 32678 raft_consensus.cc:2229] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.678522 32678 raft_consensus.cc:2258] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.680202 32678 ts_tablet_manager.cc:1905] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.687260 31937 catalog_manager.cc:6565] ReplaceTablet: tablet e605554fc4174a44acf37672f4a0fd76 deleted and replaced by tablet 71aad93fcc8b42e6b55db6b3ed6b4f91
I20260809 14:29:55.687645 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet e605554fc4174a44acf37672f4a0fd76 with delete_type TABLET_DATA_DELETED (replaced by tablet 71aad93fcc8b42e6b55db6b3ed6b4f91) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:55.688480 32677 tablet_replica.cc:330] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:55.689150 32677 raft_consensus.cc:2229] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.689622 32677 raft_consensus.cc:2258] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.688195 32306 tablet_service.cc:1467] Processing CreateTablet for tablet dca78fdbf9ba41c9889903b58898e94d (DEFAULT_TABLE table=default.Table_dadc1d59ab5f4b799d8d5d1cf23a3580 [id=0b1913c09d3342d3a9f79d3984530c94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.691032 32306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca78fdbf9ba41c9889903b58898e94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.658599 32691 consensus_queue.cc:260] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.666646 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet 664f49ed04144a59a8fd6dd1e61a6907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:55.692368 32677 ts_tablet_manager.cc:1905] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.707903 32691 ts_tablet_manager.cc:1428] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.125s user 0.006s sys 0.002s
I20260809 14:29:55.667644 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet 664f49ed04144a59a8fd6dd1e61a6907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:55.711798 32705 raft_consensus.cc:374] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.712203 32705 raft_consensus.cc:729] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:55.721522 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet e605554fc4174a44acf37672f4a0fd76 with delete_type TABLET_DATA_DELETED (replaced by tablet 71aad93fcc8b42e6b55db6b3ed6b4f91) from {username='slave'} at 127.0.0.1:41544
W20260809 14:29:55.723361 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 664f49ed04144a59a8fd6dd1e61a6907 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 664f49ed04144a59a8fd6dd1e61a6907 already in progress: creating tablet
I20260809 14:29:55.724252 32141 tablet_service.cc:1467] Processing CreateTablet for tablet dca78fdbf9ba41c9889903b58898e94d (DEFAULT_TABLE table=default.Table_dadc1d59ab5f4b799d8d5d1cf23a3580 [id=0b1913c09d3342d3a9f79d3984530c94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.725515 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca78fdbf9ba41c9889903b58898e94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.725327 32490 tablet_service.cc:1467] Processing CreateTablet for tablet c468382a28934d50babf7bbc9a834c08 (DEFAULT_TABLE table=default.Table_3d42eec9be6d469dbdede7cd0553e019 [id=44f0605ad9e4418ca64ceca2a4120cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.728781 32676 tablet_replica.cc:330] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:55.728780 32486 tablet_service.cc:1467] Processing CreateTablet for tablet dca78fdbf9ba41c9889903b58898e94d (DEFAULT_TABLE table=default.Table_dadc1d59ab5f4b799d8d5d1cf23a3580 [id=0b1913c09d3342d3a9f79d3984530c94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.729477 32676 raft_consensus.cc:2229] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:55.730199 32676 pending_rounds.cc:62] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:29:55.730404 32676 pending_rounds.cc:68] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626617870118912 op_type: NO_OP noop_request { }
I20260809 14:29:55.730756 32676 raft_consensus.cc:2866] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:55.731062 32676 raft_consensus.cc:2258] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.731664 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c468382a28934d50babf7bbc9a834c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.734133 32676 ts_tablet_manager.cc:1905] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.733867 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 969849d30c204937a7508e8d54dae427 (DEFAULT_TABLE table=default.Table_a8d4268684564e2e988b1e2bc08f779c [id=750e5ee924cc4aa0a577180b7ec93367]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.735590 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969849d30c204937a7508e8d54dae427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.737172 32143 tablet_service.cc:1467] Processing CreateTablet for tablet c468382a28934d50babf7bbc9a834c08 (DEFAULT_TABLE table=default.Table_3d42eec9be6d469dbdede7cd0553e019 [id=44f0605ad9e4418ca64ceca2a4120cf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.738212 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c468382a28934d50babf7bbc9a834c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.740818 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca78fdbf9ba41c9889903b58898e94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.744526 32656 raft_consensus.cc:482] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:55.744976 32656 raft_consensus.cc:504] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.745939 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3e6508d161504819b3fd60b6def8a4b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:55.746861 32656 leader_election.cc:288] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
W20260809 14:29:55.747644 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e6508d161504819b3fd60b6def8a4b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.749069 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5951ffedd0cd481f9f425f0af3b8ad01" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:55.750177 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5951ffedd0cd481f9f425f0af3b8ad01" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:29:55.752290 32260 leader_election.cc:341] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:29:55.752908 32260 leader_election.cc:341] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:55.753274 32260 leader_election.cc:302] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:29:55.755970 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13f6c6a2ff894ec3a795f0c3f044ec17 has no consensus state
I20260809 14:29:55.757593 32656 raft_consensus.cc:2726] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:55.758435 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 84c8eb8ad2dd406085779ed6a86eb70e (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.758594 32485 tablet_service.cc:1467] Processing CreateTablet for tablet 969849d30c204937a7508e8d54dae427 (DEFAULT_TABLE table=default.Table_a8d4268684564e2e988b1e2bc08f779c [id=750e5ee924cc4aa0a577180b7ec93367]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.759531 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c8eb8ad2dd406085779ed6a86eb70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.759872 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969849d30c204937a7508e8d54dae427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.761822 32693 tablet_bootstrap.cc:492] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:55.762212 32693 ts_tablet_manager.cc:1397] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.159s user 0.003s sys 0.008s
I20260809 14:29:55.764954 32693 raft_consensus.cc:348] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.765684 32693 raft_consensus.cc:374] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.765970 32693 raft_consensus.cc:729] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:55.766649 32693 consensus_queue.cc:260] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.772739 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 969849d30c204937a7508e8d54dae427 (DEFAULT_TABLE table=default.Table_a8d4268684564e2e988b1e2bc08f779c [id=750e5ee924cc4aa0a577180b7ec93367]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.774478 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969849d30c204937a7508e8d54dae427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.774243 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ade4d74e855d41939005da22dbe0cde9" } modify_external_catalogs: true
I20260809 14:29:55.774816 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ade4d74e855d41939005da22dbe0cde9" } modify_external_catalogs: true
I20260809 14:29:55.778288 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 84c8eb8ad2dd406085779ed6a86eb70e (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.779315 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 5951ffedd0cd481f9f425f0af3b8ad01 (table default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]). Expected version 2 got 0
I20260809 14:29:55.779632 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c8eb8ad2dd406085779ed6a86eb70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.798391 32691 tablet_bootstrap.cc:492] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:55.807413 32705 consensus_queue.cc:260] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.818186 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 84c8eb8ad2dd406085779ed6a86eb70e (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.820739 32693 ts_tablet_manager.cc:1428] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.058s user 0.003s sys 0.003s
I20260809 14:29:55.824944 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c8eb8ad2dd406085779ed6a86eb70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.830487 32677 ts_tablet_manager.cc:1918] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:55.825099 32678 ts_tablet_manager.cc:1918] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:55.838095 32678 log.cc:1196] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/e605554fc4174a44acf37672f4a0fd76
I20260809 14:29:55.838788 32677 log.cc:1196] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/e605554fc4174a44acf37672f4a0fd76
I20260809 14:29:55.839617 32678 ts_tablet_manager.cc:1939] T e605554fc4174a44acf37672f4a0fd76 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:55.842403 32705 ts_tablet_manager.cc:1428] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.194s user 0.005s sys 0.001s
W20260809 14:29:55.845160 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 849d798165144531810a03dc2b9b5f1e consensus state has no leader
I20260809 14:29:55.858582 31940 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet e89340cf1b6b4057a468334c971a20da
I20260809 14:29:55.841099 32677 ts_tablet_manager.cc:1939] T e605554fc4174a44acf37672f4a0fd76 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:55.852767 32693 tablet_bootstrap.cc:492] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:55.863857 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_733ab1fa87b647d2961d60a43900c952" } modify_external_catalogs: true
I20260809 14:29:55.864411 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_733ab1fa87b647d2961d60a43900c952" } modify_external_catalogs: true
I20260809 14:29:55.864563 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet e605554fc4174a44acf37672f4a0fd76 (table default.Table_5680c3dd85d14f089c9ef93afaf56372 [id=032f6264a2c842f1ba6b007331ccfcc6]) successfully deleted
I20260809 14:29:55.865952 32691 tablet_bootstrap.cc:654] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.824381 32676 ts_tablet_manager.cc:1918] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:55.875948 32676 log.cc:1196] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/e605554fc4174a44acf37672f4a0fd76
I20260809 14:29:55.882233 31948 master_service.cc:946] ReplaceTablet: received request to replace tablet 6e873f3b715c424e8693dbcba0990fad from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:55.876729 32676 ts_tablet_manager.cc:1939] T e605554fc4174a44acf37672f4a0fd76 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:55.883080 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet 664f49ed04144a59a8fd6dd1e61a6907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:55.887800 32693 tablet_bootstrap.cc:654] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:55.896688 32655 raft_consensus.cc:482] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:55.897156 32655 raft_consensus.cc:504] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.900375 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet 049678aadd4549d1b831174727647ec8 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:55.902617 32678 tablet_replica.cc:330] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:55.903205 32678 raft_consensus.cc:2229] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.903609 32678 raft_consensus.cc:2258] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.903434 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "664f49ed04144a59a8fd6dd1e61a6907" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:55.904269 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "664f49ed04144a59a8fd6dd1e61a6907" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:55.904881 32170 raft_consensus.cc:2454] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
W20260809 14:29:55.905910 32443 leader_election.cc:341] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:55.907936 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet e605554fc4174a44acf37672f4a0fd76 (table default.Table_5680c3dd85d14f089c9ef93afaf56372 [id=032f6264a2c842f1ba6b007331ccfcc6]) successfully deleted
I20260809 14:29:55.908689 32676 tablet_replica.cc:330] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:55.909312 32676 raft_consensus.cc:2229] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:55.909588 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet e605554fc4174a44acf37672f4a0fd76 (table default.Table_5680c3dd85d14f089c9ef93afaf56372 [id=032f6264a2c842f1ba6b007331ccfcc6]) successfully deleted
I20260809 14:29:55.909845 32676 raft_consensus.cc:2258] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:55.910794 32705 tablet_bootstrap.cc:492] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:55.907064 32439 leader_election.cc:302] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
W20260809 14:29:55.918713 32439 raft_consensus.cc:2670] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:29:55.918257 32655 leader_election.cc:288] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:55.922851 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_1b45941b989d42779572c3500879e62c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:55.924630 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b45941b989d42779572c3500879e62c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.929777 32676 ts_tablet_manager.cc:1905] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.932426 32678 ts_tablet_manager.cc:1905] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:55.935138 32693 tablet_bootstrap.cc:492] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:55.935570 32693 ts_tablet_manager.cc:1397] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.092s user 0.013s sys 0.006s
I20260809 14:29:55.938344 32693 raft_consensus.cc:348] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.939069 32693 raft_consensus.cc:374] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:55.939342 32693 raft_consensus.cc:729] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:55.954130 31937 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]). Expected version 1 got 0
I20260809 14:29:55.956595 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (DEFAULT_TABLE table=default.Table_379bcab3d28446218fcd18205d6bcea0 [id=20d2073f354649b3b4bc4d583660c93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.957269 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (DEFAULT_TABLE table=default.Table_379bcab3d28446218fcd18205d6bcea0 [id=20d2073f354649b3b4bc4d583660c93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.957839 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13f6c6a2ff894ec3a795f0c3f044ec17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.958504 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13f6c6a2ff894ec3a795f0c3f044ec17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.956218 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (DEFAULT_TABLE table=default.Table_379bcab3d28446218fcd18205d6bcea0 [id=20d2073f354649b3b4bc4d583660c93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.960084 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13f6c6a2ff894ec3a795f0c3f044ec17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.961537 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_379bcab3d28446218fcd18205d6bcea0" } new_table_name: "default.Table_82bc1bf6e0f04a4680201cc8e2704244" modify_external_catalogs: true
I20260809 14:29:55.962512 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3677aa9bbc744eb59b538d1d53adc9e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:55.964579 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3677aa9bbc744eb59b538d1d53adc9e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:55.966198 32705 tablet_bootstrap.cc:654] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
W20260809 14:29:55.968012 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (table default.Table_379bcab3d28446218fcd18205d6bcea0 [id=20d2073f354649b3b4bc4d583660c93a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 13f6c6a2ff894ec3a795f0c3f044ec17 consensus state has no leader
I20260809 14:29:55.969676 32693 consensus_queue.cc:260] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:55.976755 32693 ts_tablet_manager.cc:1428] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.041s user 0.007s sys 0.001s
I20260809 14:29:55.983165 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 (DEFAULT_TABLE table=default.Table_68743f87e3364e37a579888759048d44 [id=c52a212b949c4c93b9f6732bbcaaeb33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.985416 32140 tablet_service.cc:1467] Processing CreateTablet for tablet a9f61933c2004a43af2c748ea445942c (DEFAULT_TABLE table=default.Table_7d6f721ab1324625bcf5ac42e3e20f8d [id=c1935d3bc49c405abe4d006932820224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.986523 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9ae593e06b4e2a949b799380c1a3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.992792 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9f61933c2004a43af2c748ea445942c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.993362 32693 tablet_bootstrap.cc:492] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:55.981642 32307 tablet_service.cc:1467] Processing CreateTablet for tablet a9f61933c2004a43af2c748ea445942c (DEFAULT_TABLE table=default.Table_7d6f721ab1324625bcf5ac42e3e20f8d [id=c1935d3bc49c405abe4d006932820224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.994633 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9f61933c2004a43af2c748ea445942c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.984115 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 (DEFAULT_TABLE table=default.Table_68743f87e3364e37a579888759048d44 [id=c52a212b949c4c93b9f6732bbcaaeb33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.997396 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9ae593e06b4e2a949b799380c1a3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.997411 32484 tablet_service.cc:1467] Processing CreateTablet for tablet a9f61933c2004a43af2c748ea445942c (DEFAULT_TABLE table=default.Table_7d6f721ab1324625bcf5ac42e3e20f8d [id=c1935d3bc49c405abe4d006932820224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:55.998579 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9f61933c2004a43af2c748ea445942c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:55.985780 32306 tablet_service.cc:1467] Processing CreateTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 (DEFAULT_TABLE table=default.Table_5680c3dd85d14f089c9ef93afaf56372 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.001242 32306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.002655 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 (DEFAULT_TABLE table=default.Table_68743f87e3364e37a579888759048d44 [id=c52a212b949c4c93b9f6732bbcaaeb33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.003672 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9ae593e06b4e2a949b799380c1a3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.015226 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 (DEFAULT_TABLE table=default.Table_5680c3dd85d14f089c9ef93afaf56372 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.016599 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.025791 32676 ts_tablet_manager.cc:1918] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:56.026127 32676 log.cc:1196] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/664f49ed04144a59a8fd6dd1e61a6907
I20260809 14:29:56.026700 32676 ts_tablet_manager.cc:1939] T 664f49ed04144a59a8fd6dd1e61a6907 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:56.034758 32691 tablet_bootstrap.cc:492] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:56.040995 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 664f49ed04144a59a8fd6dd1e61a6907 (table default.Table_8580644785dc4eaab7fefc3d44b66089 [id=a50d475ae87f4c1ebe36017da6ac772e]) successfully deleted
I20260809 14:29:56.045864 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_13a5f51ccd244dc5a0250c3babc6e411"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:56.046797 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 664f49ed04144a59a8fd6dd1e61a6907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:56.047833 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 6e873f3b715c424e8693dbcba0990fad
W20260809 14:29:56.047928 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13a5f51ccd244dc5a0250c3babc6e411 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:56.048913 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_1b45941b989d42779572c3500879e62c" } new_table_name: "default.Table_18bbb19abeca4f54a421b38edeba896c" modify_external_catalogs: true
I20260809 14:29:56.050021 32677 tablet_replica.cc:330] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:56.049479 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_acb53b3bc5b24b989ccf44b01709c500"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:56.050683 32677 raft_consensus.cc:2229] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:56.051097 32677 raft_consensus.cc:2258] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 14:29:56.051594 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acb53b3bc5b24b989ccf44b01709c500 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:56.053452 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e873f3b715c424e8693dbcba0990fad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:56.055344 32691 ts_tablet_manager.cc:1397] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.263s user 0.009s sys 0.004s
I20260809 14:29:56.056442 32677 ts_tablet_manager.cc:1905] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:56.056563 32693 tablet_bootstrap.cc:654] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.037377 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 (DEFAULT_TABLE table=default.Table_5680c3dd85d14f089c9ef93afaf56372 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.057603 32485 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e873f3b715c424e8693dbcba0990fad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:56.061537 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 049678aadd4549d1b831174727647ec8
I20260809 14:29:56.062984 31942 catalog_manager.cc:6565] ReplaceTablet: tablet 049678aadd4549d1b831174727647ec8 deleted and replaced by tablet ce085227492e49549d3304a05a66a27d
I20260809 14:29:56.060704 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.070930 32305 tablet_service.cc:1514] Processing DeleteTablet for tablet 049678aadd4549d1b831174727647ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet ce085227492e49549d3304a05a66a27d) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:56.071961 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e873f3b715c424e8693dbcba0990fad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:29:56.061765 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 6e873f3b715c424e8693dbcba0990fad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e873f3b715c424e8693dbcba0990fad already in progress: creating tablet
I20260809 14:29:56.077066 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet 049678aadd4549d1b831174727647ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet ce085227492e49549d3304a05a66a27d) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:56.079221 32691 raft_consensus.cc:348] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
W20260809 14:29:56.089246 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 049678aadd4549d1b831174727647ec8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 049678aadd4549d1b831174727647ec8 already in progress: creating tablet
I20260809 14:29:56.099627 32705 tablet_bootstrap.cc:492] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:56.084738 32485 tablet_service.cc:1514] Processing DeleteTablet for tablet 049678aadd4549d1b831174727647ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet ce085227492e49549d3304a05a66a27d) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:56.102332 32691 raft_consensus.cc:374] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.103156 32691 raft_consensus.cc:729] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.103926 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_68743f87e3364e37a579888759048d44" } modify_external_catalogs: true
I20260809 14:29:56.104852 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_68743f87e3364e37a579888759048d44" } modify_external_catalogs: true
I20260809 14:29:56.120134 32691 consensus_queue.cc:260] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.125190 32705 ts_tablet_manager.cc:1397] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.215s user 0.009s sys 0.005s
I20260809 14:29:56.135691 32705 raft_consensus.cc:348] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
W20260809 14:29:56.149544 31938 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13f6c6a2ff894ec3a795f0c3f044ec17 consensus state has no leader
I20260809 14:29:56.149775 32691 ts_tablet_manager.cc:1428] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.092s user 0.003s sys 0.003s
I20260809 14:29:56.172082 32691 tablet_bootstrap.cc:492] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:56.107923 32678 ts_tablet_manager.cc:1918] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:56.178200 32678 log.cc:1196] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/664f49ed04144a59a8fd6dd1e61a6907
I20260809 14:29:56.179190 32678 ts_tablet_manager.cc:1939] T 664f49ed04144a59a8fd6dd1e61a6907 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:56.188982 32693 tablet_bootstrap.cc:492] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:56.189572 32693 ts_tablet_manager.cc:1397] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.196s user 0.017s sys 0.013s
I20260809 14:29:56.193135 32693 raft_consensus.cc:348] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.194056 32693 raft_consensus.cc:374] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.194613 32693 raft_consensus.cc:729] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.195416 32693 consensus_queue.cc:260] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.196962 32705 raft_consensus.cc:374] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.199805 32677 ts_tablet_manager.cc:1918] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:56.200995 32705 raft_consensus.cc:729] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.206609 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f72f5e4a712b4a519089aa13f2060a2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:56.210563 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 664f49ed04144a59a8fd6dd1e61a6907 (table default.Table_8580644785dc4eaab7fefc3d44b66089 [id=a50d475ae87f4c1ebe36017da6ac772e]) successfully deleted
W20260809 14:29:56.211851 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 6e873f3b715c424e8693dbcba0990fad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e873f3b715c424e8693dbcba0990fad already in progress: creating tablet
W20260809 14:29:56.212411 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f72f5e4a712b4a519089aa13f2060a2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:29:56.212611 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 049678aadd4549d1b831174727647ec8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 049678aadd4549d1b831174727647ec8 already in progress: creating tablet
I20260809 14:29:56.207319 32705 consensus_queue.cc:260] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.229984 32705 ts_tablet_manager.cc:1428] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.097s user 0.004s sys 0.003s
I20260809 14:29:56.252140 31946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3b3fded95fa74c4086529b42834a0955" } modify_external_catalogs: true
I20260809 14:29:56.252833 31946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3b3fded95fa74c4086529b42834a0955" } modify_external_catalogs: true
I20260809 14:29:56.258859 32705 tablet_bootstrap.cc:492] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:56.259209 32677 log.cc:1196] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/664f49ed04144a59a8fd6dd1e61a6907
I20260809 14:29:56.204363 32693 ts_tablet_manager.cc:1428] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.014s user 0.004s sys 0.003s
I20260809 14:29:56.263525 32693 tablet_bootstrap.cc:492] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:56.271675 32691 tablet_bootstrap.cc:654] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.275566 31943 master_service.cc:946] ReplaceTablet: received request to replace tablet 5951ffedd0cd481f9f425f0af3b8ad01 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:56.279306 32693 tablet_bootstrap.cc:654] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.299065 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 049678aadd4549d1b831174727647ec8 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:56.301052 32677 ts_tablet_manager.cc:1939] T 664f49ed04144a59a8fd6dd1e61a6907 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:56.302847 32705 tablet_bootstrap.cc:654] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.304934 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 664f49ed04144a59a8fd6dd1e61a6907 (table default.Table_8580644785dc4eaab7fefc3d44b66089 [id=a50d475ae87f4c1ebe36017da6ac772e]) successfully deleted
I20260809 14:29:56.306360 32691 tablet_bootstrap.cc:492] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:56.306770 32691 ts_tablet_manager.cc:1397] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.135s user 0.009s sys 0.005s
W20260809 14:29:56.309067 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 57c074cef80146e28a3f3795077d047a has no consensus state
I20260809 14:29:56.321300 32691 raft_consensus.cc:348] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.337865 32691 raft_consensus.cc:374] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.338689 32691 raft_consensus.cc:729] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.345621 32691 consensus_queue.cc:260] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.349196 31937 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]). Expected version 1 got 0
I20260809 14:29:56.352659 31948 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]). Expected version 1 got 0
I20260809 14:29:56.353811 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 57c074cef80146e28a3f3795077d047a (DEFAULT_TABLE table=default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.354894 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c074cef80146e28a3f3795077d047a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.356767 31940 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]). Expected version 1 got 0
I20260809 14:29:56.358114 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 57c074cef80146e28a3f3795077d047a (DEFAULT_TABLE table=default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.359382 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c074cef80146e28a3f3795077d047a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.363464 32139 tablet_service.cc:1467] Processing CreateTablet for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c (DEFAULT_TABLE table=default.Table_3e6508d161504819b3fd60b6def8a4b0 [id=11d64f84c8854864b5ea0f591af8d79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.364553 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.365044 32487 tablet_service.cc:1467] Processing CreateTablet for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c (DEFAULT_TABLE table=default.Table_3e6508d161504819b3fd60b6def8a4b0 [id=11d64f84c8854864b5ea0f591af8d79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.365401 31948 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]). Expected version 1 got 0
I20260809 14:29:56.366135 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.368391 32306 tablet_service.cc:1467] Processing CreateTablet for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c (DEFAULT_TABLE table=default.Table_3e6508d161504819b3fd60b6def8a4b0 [id=11d64f84c8854864b5ea0f591af8d79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.368940 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_13a5f51ccd244dc5a0250c3babc6e411" } new_table_name: "default.Table_3ecd77174e9948a59988f0b186dc8793" modify_external_catalogs: true
I20260809 14:29:56.369625 32306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.373704 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5352320f0eb24ca5a85ceb658929954a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:56.375422 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5352320f0eb24ca5a85ceb658929954a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:56.384718 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 01cfc9e0319748c8b0e928fc64706933 (DEFAULT_TABLE table=default.Table_3b3fded95fa74c4086529b42834a0955 [id=f383923a235841a4a476b6cf31d33ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.385058 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 57c074cef80146e28a3f3795077d047a (DEFAULT_TABLE table=default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.385067 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 01cfc9e0319748c8b0e928fc64706933 (DEFAULT_TABLE table=default.Table_3b3fded95fa74c4086529b42834a0955 [id=f383923a235841a4a476b6cf31d33ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.385862 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cfc9e0319748c8b0e928fc64706933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.386129 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c074cef80146e28a3f3795077d047a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.386445 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cfc9e0319748c8b0e928fc64706933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.391417 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]). Expected version 1 got 0
I20260809 14:29:56.392246 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]). Expected version 1 got 0
I20260809 14:29:56.393105 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 01cfc9e0319748c8b0e928fc64706933 (DEFAULT_TABLE table=default.Table_3b3fded95fa74c4086529b42834a0955 [id=f383923a235841a4a476b6cf31d33ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:56.394443 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cfc9e0319748c8b0e928fc64706933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:56.396862 31938 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]). Expected version 1 got 0
I20260809 14:29:56.404559 32691 ts_tablet_manager.cc:1428] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.098s user 0.008s sys 0.001s
I20260809 14:29:56.412859 32693 tablet_bootstrap.cc:492] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:56.413352 32693 ts_tablet_manager.cc:1397] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.150s user 0.011s sys 0.002s
W20260809 14:29:56.415521 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 049678aadd4549d1b831174727647ec8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 049678aadd4549d1b831174727647ec8 already in progress: creating tablet
I20260809 14:29:56.415889 32693 raft_consensus.cc:348] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.416648 32693 raft_consensus.cc:374] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.416989 32693 raft_consensus.cc:729] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:29:56.417878 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 6e873f3b715c424e8693dbcba0990fad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e873f3b715c424e8693dbcba0990fad already in progress: creating tablet
I20260809 14:29:56.422012 32691 tablet_bootstrap.cc:492] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:56.422330 32705 tablet_bootstrap.cc:492] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:56.423115 32705 ts_tablet_manager.cc:1397] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.164s user 0.011s sys 0.006s
I20260809 14:29:56.425434 32655 raft_consensus.cc:482] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:56.425841 32655 raft_consensus.cc:504] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.427536 32655 leader_election.cc:288] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:56.430413 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]). Expected version 1 got 0
I20260809 14:29:56.431341 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "765d102c8c6444b89da084f27a4cf8ed" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:56.432018 32333 raft_consensus.cc:2454] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:56.431259 32693 consensus_queue.cc:260] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.440904 32443 leader_election.cc:302] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:56.441826 32693 ts_tablet_manager.cc:1428] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20260809 14:29:56.442409 32655 raft_consensus.cc:2781] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:56.442755 32693 tablet_bootstrap.cc:492] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:56.442809 32655 raft_consensus.cc:482] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:56.444288 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 049678aadd4549d1b831174727647ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet ce085227492e49549d3304a05a66a27d) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:56.446229 32677 tablet_replica.cc:330] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:56.447295 32677 raft_consensus.cc:2229] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:56.440182 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "765d102c8c6444b89da084f27a4cf8ed" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:56.453747 32655 raft_consensus.cc:3037] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.448046 32677 raft_consensus.cc:2258] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:56.455462 32705 raft_consensus.cc:348] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
W20260809 14:29:56.456811 32439 leader_election.cc:341] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:56.456240 32705 raft_consensus.cc:374] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.461987 32655 raft_consensus.cc:504] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.461500 32705 raft_consensus.cc:729] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.465152 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "765d102c8c6444b89da084f27a4cf8ed" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:56.465541 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f7fd4627a85c4f358f737fad5077415c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:56.466061 32705 consensus_queue.cc:260] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.466902 32170 raft_consensus.cc:3037] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.471104 32655 leader_election.cc:288] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:56.473724 32170 raft_consensus.cc:2454] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:56.476171 32691 tablet_bootstrap.cc:654] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.477332 32439 leader_election.cc:302] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:56.478430 32655 raft_consensus.cc:2781] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
W20260809 14:29:56.478616 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7fd4627a85c4f358f737fad5077415c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:56.490631 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "765d102c8c6444b89da084f27a4cf8ed" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:56.491238 32333 raft_consensus.cc:3037] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.493242 32693 tablet_bootstrap.cc:654] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.499051 32333 raft_consensus.cc:2454] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:56.491261 32677 ts_tablet_manager.cc:1905] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:56.520910 32655 raft_consensus.cc:686] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:29:56.522571 32655 consensus_queue.cc:237] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.529822 32693 tablet_bootstrap.cc:492] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:56.530251 32693 ts_tablet_manager.cc:1397] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.088s user 0.013s sys 0.004s
I20260809 14:29:56.532863 32693 raft_consensus.cc:348] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.533608 32693 raft_consensus.cc:374] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.533932 32693 raft_consensus.cc:729] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.534888 32693 consensus_queue.cc:260] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.537127 32705 ts_tablet_manager.cc:1428] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.088s user 0.005s sys 0.001s
I20260809 14:29:56.542554 32693 ts_tablet_manager.cc:1428] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.012s user 0.004s sys 0.002s
I20260809 14:29:56.544554 32705 tablet_bootstrap.cc:492] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:56.557632 32693 tablet_bootstrap.cc:492] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:56.558758 32691 tablet_bootstrap.cc:492] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:56.559286 32691 ts_tablet_manager.cc:1397] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.137s user 0.013s sys 0.006s
I20260809 14:29:56.564473 32655 raft_consensus.cc:482] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:56.564970 32655 raft_consensus.cc:504] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.571559 32705 tablet_bootstrap.cc:654] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.573213 32691 raft_consensus.cc:348] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.573966 32691 raft_consensus.cc:374] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.574712 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48a1586a9164629a7dfe057bbf12dc0" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:56.575562 32333 raft_consensus.cc:2454] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:56.576997 32443 leader_election.cc:302] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:56.577986 32757 raft_consensus.cc:2781] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:56.578434 32757 raft_consensus.cc:482] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:56.574237 32691 raft_consensus.cc:729] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.579015 32757 raft_consensus.cc:3037] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.580097 32691 consensus_queue.cc:260] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.588337 32677 ts_tablet_manager.cc:1918] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:56.591483 32691 ts_tablet_manager.cc:1428] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.032s user 0.008s sys 0.000s
I20260809 14:29:56.537123 31946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 01cfc9e0319748c8b0e928fc64706933
I20260809 14:29:56.596853 32655 leader_election.cc:288] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:56.591341 32693 tablet_bootstrap.cc:654] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.599792 32677 log.cc:1196] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/049678aadd4549d1b831174727647ec8
I20260809 14:29:56.559752 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 5951ffedd0cd481f9f425f0af3b8ad01
I20260809 14:29:56.561004 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 8b9ae593e06b4e2a949b799380c1a3e3
I20260809 14:29:56.602289 32691 tablet_bootstrap.cc:492] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:56.602257 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet 13f6c6a2ff894ec3a795f0c3f044ec17 from {username='slave'} at 127.0.0.1:49782
W20260809 14:29:56.567668 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e378d7763c5046249011a6d6e0617c97 has no consensus state
I20260809 14:29:56.605229 31943 catalog_manager.cc:6565] ReplaceTablet: tablet 5951ffedd0cd481f9f425f0af3b8ad01 deleted and replaced by tablet 609b3e6049ec4b0582d78037b7416710
I20260809 14:29:56.605981 32677 ts_tablet_manager.cc:1939] T 049678aadd4549d1b831174727647ec8 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:56.574021 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3e6508d161504819b3fd60b6def8a4b0" } new_table_name: "default.Table_331f2b568f3b4937b41ddb58af580e97" modify_external_catalogs: true
I20260809 14:29:56.601754 32757 raft_consensus.cc:504] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.575665 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f3f904c0b94440ddb1f6f7f733689d8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:56.576862 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_4adc37cd84f84f3fb4f7347c01a0e498"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:56.578101 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]). Expected version 1 got 0
I20260809 14:29:56.617026 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:56.617983 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet 5951ffedd0cd481f9f425f0af3b8ad01 with delete_type TABLET_DATA_DELETED (replaced by tablet 609b3e6049ec4b0582d78037b7416710) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:56.616781 32140 tablet_service.cc:1514] Processing DeleteTablet for tablet 01cfc9e0319748c8b0e928fc64706933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:56.594064 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48a1586a9164629a7dfe057bbf12dc0" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:56.618860 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 5951ffedd0cd481f9f425f0af3b8ad01 with delete_type TABLET_DATA_DELETED (replaced by tablet 609b3e6049ec4b0582d78037b7416710) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:56.619151 32170 raft_consensus.cc:2454] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:56.619616 32678 tablet_replica.cc:330] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:56.620326 32678 raft_consensus.cc:2229] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:56.620788 32678 raft_consensus.cc:2258] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:56.621173 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 01cfc9e0319748c8b0e928fc64706933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:56.621780 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48a1586a9164629a7dfe057bbf12dc0" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:56.622481 32333 raft_consensus.cc:3037] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.622314 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:56.625285 32678 ts_tablet_manager.cc:1905] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:56.625213 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 049678aadd4549d1b831174727647ec8 (table default.Table_7d8d4891be3a46e8adc1a2074020612c [id=1240bfe259ae4a2596755c9b3d3d7474]) successfully deleted
W20260809 14:29:56.626066 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 8b9ae593e06b4e2a949b799380c1a3e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b9ae593e06b4e2a949b799380c1a3e3 already in progress: creating tablet
I20260809 14:29:56.626346 32691 tablet_bootstrap.cc:654] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:29:56.626626 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 5951ffedd0cd481f9f425f0af3b8ad01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5951ffedd0cd481f9f425f0af3b8ad01 already in progress: creating tablet
W20260809 14:29:56.627156 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 01cfc9e0319748c8b0e928fc64706933 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01cfc9e0319748c8b0e928fc64706933 already in progress: creating tablet
W20260809 14:29:56.628077 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3f904c0b94440ddb1f6f7f733689d8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:56.629493 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 01cfc9e0319748c8b0e928fc64706933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:56.630743 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:56.632099 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 5951ffedd0cd481f9f425f0af3b8ad01 with delete_type TABLET_DATA_DELETED (replaced by tablet 609b3e6049ec4b0582d78037b7416710) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:56.635070 31945 catalog_manager.cc:5495] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 1 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } }
I20260809 14:29:56.636399 31945 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]). Expected version 1 got 0
W20260809 14:29:56.638362 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4adc37cd84f84f3fb4f7347c01a0e498 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:56.636288 32333 raft_consensus.cc:2454] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:56.645670 32443 leader_election.cc:302] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:56.646991 32655 raft_consensus.cc:2781] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:56.649847 32693 tablet_bootstrap.cc:492] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:56.650233 32693 ts_tablet_manager.cc:1397] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.093s user 0.005s sys 0.009s
I20260809 14:29:56.650883 32655 raft_consensus.cc:686] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:29:56.651813 32655 consensus_queue.cc:237] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.653294 32693 raft_consensus.cc:348] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.653975 32693 raft_consensus.cc:374] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.654209 32693 raft_consensus.cc:729] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.654784 32693 consensus_queue.cc:260] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:56.660979 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 01cfc9e0319748c8b0e928fc64706933 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01cfc9e0319748c8b0e928fc64706933 already in progress: creating tablet
I20260809 14:29:56.662580 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48a1586a9164629a7dfe057bbf12dc0" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:56.663353 32170 raft_consensus.cc:3037] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.666325 32691 tablet_bootstrap.cc:492] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:56.666707 32691 ts_tablet_manager.cc:1397] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.065s user 0.011s sys 0.000s
W20260809 14:29:56.669298 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 8b9ae593e06b4e2a949b799380c1a3e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b9ae593e06b4e2a949b799380c1a3e3 already in progress: creating tablet
I20260809 14:29:56.670228 32691 raft_consensus.cc:348] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.670847 32691 raft_consensus.cc:374] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.671022 32757 leader_election.cc:288] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:56.671126 32691 raft_consensus.cc:729] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.673002 32170 raft_consensus.cc:2454] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:56.673816 32677 tablet_replica.cc:330] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:56.674897 32693 ts_tablet_manager.cc:1428] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.024s user 0.002s sys 0.006s
I20260809 14:29:56.676599 32693 tablet_bootstrap.cc:492] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:56.677030 32691 consensus_queue.cc:260] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.679095 32691 ts_tablet_manager.cc:1428] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260809 14:29:56.679937 32691 tablet_bootstrap.cc:492] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:56.695798 32656 raft_consensus.cc:482] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:56.696259 32656 raft_consensus.cc:504] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.697314 32677 raft_consensus.cc:2229] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:56.697877 32677 raft_consensus.cc:2258] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:56.698230 32656 leader_election.cc:288] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:56.700300 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e873f3b715c424e8693dbcba0990fad" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:56.701151 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e873f3b715c424e8693dbcba0990fad" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:56.701623 32693 tablet_bootstrap.cc:654] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.701865 32515 raft_consensus.cc:2454] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:29:56.703228 32691 tablet_bootstrap.cc:654] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.703491 32260 leader_election.cc:302] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:56.704309 32656 raft_consensus.cc:2781] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:56.704849 32656 raft_consensus.cc:482] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:56.705289 32656 raft_consensus.cc:3037] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.713197 32677 ts_tablet_manager.cc:1905] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:56.714543 32705 tablet_bootstrap.cc:492] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:56.715058 32705 ts_tablet_manager.cc:1397] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.171s user 0.005s sys 0.005s
I20260809 14:29:56.717137 32678 ts_tablet_manager.cc:1918] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:56.717490 32678 log.cc:1196] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/5951ffedd0cd481f9f425f0af3b8ad01
I20260809 14:29:56.718086 32678 ts_tablet_manager.cc:1939] T 5951ffedd0cd481f9f425f0af3b8ad01 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:56.722916 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 5951ffedd0cd481f9f425f0af3b8ad01 (table default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]) successfully deleted
I20260809 14:29:56.723557 32693 tablet_bootstrap.cc:492] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:56.724005 32693 ts_tablet_manager.cc:1397] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.000s
I20260809 14:29:56.726840 32693 raft_consensus.cc:348] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.727821 32693 raft_consensus.cc:374] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 14:29:56.732404 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 01cfc9e0319748c8b0e928fc64706933 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01cfc9e0319748c8b0e928fc64706933 already in progress: creating tablet
W20260809 14:29:56.733520 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 8b9ae593e06b4e2a949b799380c1a3e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b9ae593e06b4e2a949b799380c1a3e3 already in progress: creating tablet
I20260809 14:29:56.734068 32705 raft_consensus.cc:348] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.735069 32705 raft_consensus.cc:374] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 14:29:56.734865 32260 leader_election.cc:341] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:56.735452 32705 raft_consensus.cc:729] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.737354 32656 raft_consensus.cc:504] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.728176 32693 raft_consensus.cc:729] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.751796 32693 consensus_queue.cc:260] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.736285 32705 consensus_queue.cc:260] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.761955 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e873f3b715c424e8693dbcba0990fad" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:56.763216 32656 leader_election.cc:288] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:56.765414 32705 ts_tablet_manager.cc:1428] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.050s user 0.002s sys 0.008s
I20260809 14:29:56.765395 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e873f3b715c424e8693dbcba0990fad" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:56.766077 32515 raft_consensus.cc:3037] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.774492 32705 tablet_bootstrap.cc:492] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:56.775239 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c0bf6a05de414660a2f0c1ea7b5a83ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:56.777310 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0bf6a05de414660a2f0c1ea7b5a83ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:56.782927 32515 raft_consensus.cc:2454] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:29:56.784183 32705 tablet_bootstrap.cc:654] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.769253 32677 ts_tablet_manager.cc:1918] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:56.802248 32677 log.cc:1196] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/5951ffedd0cd481f9f425f0af3b8ad01
I20260809 14:29:56.802915 32677 ts_tablet_manager.cc:1939] T 5951ffedd0cd481f9f425f0af3b8ad01 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:56.805462 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 5951ffedd0cd481f9f425f0af3b8ad01 (table default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]) successfully deleted
W20260809 14:29:56.772804 32260 leader_election.cc:341] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:56.776227 302 raft_consensus.cc:482] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:56.807271 302 raft_consensus.cc:504] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.810127 302 leader_election.cc:288] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:56.777803 32693 ts_tablet_manager.cc:1428] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.054s user 0.010s sys 0.000s
I20260809 14:29:56.811218 32693 tablet_bootstrap.cc:492] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:56.813979 32691 tablet_bootstrap.cc:492] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:56.814618 32691 ts_tablet_manager.cc:1397] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.135s user 0.014s sys 0.017s
W20260809 14:29:56.819038 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 46251us
I20260809 14:29:56.818688 32693 tablet_bootstrap.cc:654] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.822520 32705 tablet_bootstrap.cc:492] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:56.822391 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9ecd2a39ee0f40cca949c529cba934ac" } modify_external_catalogs: true
I20260809 14:29:56.822916 32705 ts_tablet_manager.cc:1397] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.049s user 0.009s sys 0.001s
I20260809 14:29:56.823029 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9ecd2a39ee0f40cca949c529cba934ac" } modify_external_catalogs: true
I20260809 14:29:56.824227 32260 leader_election.cc:302] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:56.824687 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658ab8e11b37419ba2176a9175bfd936" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:56.827433 32705 raft_consensus.cc:348] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.828069 32705 raft_consensus.cc:374] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.828306 32705 raft_consensus.cc:729] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.829056 32705 consensus_queue.cc:260] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.831118 302 raft_consensus.cc:2781] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:56.830880 32691 raft_consensus.cc:348] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.831522 32691 raft_consensus.cc:374] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.831516 302 raft_consensus.cc:686] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:29:56.831764 32691 raft_consensus.cc:729] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.832587 302 consensus_queue.cc:237] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.836324 32705 ts_tablet_manager.cc:1428] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260809 14:29:56.841426 32691 consensus_queue.cc:260] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:56.846341 32260 leader_election.cc:341] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:56.849534 32705 tablet_bootstrap.cc:492] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:56.851951 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658ab8e11b37419ba2176a9175bfd936" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:56.858358 302 raft_consensus.cc:482] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:56.858744 302 raft_consensus.cc:504] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.860134 32693 tablet_bootstrap.cc:492] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:56.860621 32693 ts_tablet_manager.cc:1397] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.007s
I20260809 14:29:56.864995 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c468382a28934d50babf7bbc9a834c08" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:29:56.866760 32260 leader_election.cc:341] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:56.869086 32693 raft_consensus.cc:348] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.869684 32693 raft_consensus.cc:374] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.869930 32693 raft_consensus.cc:729] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.870551 32693 consensus_queue.cc:260] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.860134 32691 ts_tablet_manager.cc:1428] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.045s user 0.007s sys 0.001s
I20260809 14:29:56.875527 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c468382a28934d50babf7bbc9a834c08" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:56.873260 302 leader_election.cc:288] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:56.868842 32515 raft_consensus.cc:2454] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:29:56.880136 32691 tablet_bootstrap.cc:492] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:29:56.880388 32260 leader_election.cc:341] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:56.880806 32260 leader_election.cc:302] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:56.881960 302 raft_consensus.cc:2726] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:56.883522 32705 tablet_bootstrap.cc:654] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.887290 32693 ts_tablet_manager.cc:1428] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.026s user 0.003s sys 0.002s
I20260809 14:29:56.888171 32693 tablet_bootstrap.cc:492] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:56.892961 32260 leader_election.cc:302] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:56.894200 301 raft_consensus.cc:2781] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:56.894630 301 raft_consensus.cc:482] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:56.895079 301 raft_consensus.cc:3037] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:56.914894 32691 tablet_bootstrap.cc:654] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.922662 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_440a4f8d04c24bf682668ec6d4d12a4d" } modify_external_catalogs: true
I20260809 14:29:56.923432 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_440a4f8d04c24bf682668ec6d4d12a4d" } modify_external_catalogs: true
W20260809 14:29:56.929391 31938 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c consensus state has no leader
I20260809 14:29:56.931613 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_7d8d4891be3a46e8adc1a2074020612c" } new_table_name: "default.Table_7b47ee94db13435cb8e6ea1c46c528f7" modify_external_catalogs: true
I20260809 14:29:56.945710 32693 tablet_bootstrap.cc:654] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:56.947096 31939 master_service.cc:946] ReplaceTablet: received request to replace tablet 01cfc9e0319748c8b0e928fc64706933 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:56.952450 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]). Expected version 1 got 0
I20260809 14:29:56.983474 32693 tablet_bootstrap.cc:492] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:56.985038 32693 ts_tablet_manager.cc:1397] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.097s user 0.014s sys 0.002s
W20260809 14:29:56.994421 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 849d798165144531810a03dc2b9b5f1e consensus state has no leader
I20260809 14:29:56.995412 32731 raft_consensus.cc:482] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:56.996030 32731 raft_consensus.cc:504] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } }
I20260809 14:29:56.987912 32693 raft_consensus.cc:348] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:56.997690 32693 raft_consensus.cc:374] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:56.997929 32693 raft_consensus.cc:729] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:56.999477 32691 tablet_bootstrap.cc:492] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:56.999909 32691 ts_tablet_manager.cc:1397] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.120s user 0.018s sys 0.000s
W20260809 14:29:57.003110 32099 leader_election.cc:341] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 5951ffedd0cd481f9f425f0af3b8ad01
I20260809 14:29:56.993676 32705 tablet_bootstrap.cc:492] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:57.005136 32705 ts_tablet_manager.cc:1397] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.156s user 0.012s sys 0.015s
I20260809 14:29:57.012221 32691 raft_consensus.cc:348] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.013805 32691 raft_consensus.cc:374] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.014103 32691 raft_consensus.cc:729] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:29:57.015074 32095 leader_election.cc:341] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 5951ffedd0cd481f9f425f0af3b8ad01
I20260809 14:29:57.014798 32691 consensus_queue.cc:260] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.015656 32095 leader_election.cc:302] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:29:57.017560 32731 leader_election.cc:288] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:57.018453 32731 raft_consensus.cc:2726] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:57.009820 32705 raft_consensus.cc:348] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.022444 32705 raft_consensus.cc:374] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.009415 301 raft_consensus.cc:504] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.029377 32693 consensus_queue.cc:260] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.040884 32693 ts_tablet_manager.cc:1428] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.056s user 0.001s sys 0.006s
I20260809 14:29:57.041754 32693 tablet_bootstrap.cc:492] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:57.042644 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658ab8e11b37419ba2176a9175bfd936" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:29:57.044507 32260 leader_election.cc:341] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:57.045641 32705 raft_consensus.cc:729] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.046152 301 leader_election.cc:288] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:57.046794 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658ab8e11b37419ba2176a9175bfd936" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:57.047379 32515 raft_consensus.cc:3037] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:57.047199 32705 consensus_queue.cc:260] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.049230 32705 ts_tablet_manager.cc:1428] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.044s user 0.005s sys 0.002s
I20260809 14:29:57.055421 32515 raft_consensus.cc:2454] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:29:57.058274 32693 tablet_bootstrap.cc:654] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.058677 32260 leader_election.cc:302] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:57.059686 301 raft_consensus.cc:2781] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:57.060509 301 raft_consensus.cc:686] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:29:57.061654 301 consensus_queue.cc:237] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.060312 32691 ts_tablet_manager.cc:1428] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.060s user 0.003s sys 0.008s
I20260809 14:29:57.066537 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet c48a1586a9164629a7dfe057bbf12dc0
W20260809 14:29:57.068117 31938 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ce085227492e49549d3304a05a66a27d has no consensus state
I20260809 14:29:57.069657 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet c48a1586a9164629a7dfe057bbf12dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:57.070436 32763 tablet_replica.cc:330] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:57.071514 32763 raft_consensus.cc:2229] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:57.072029 32763 raft_consensus.cc:2258] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:57.073388 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet c48a1586a9164629a7dfe057bbf12dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:57.074123 32678 tablet_replica.cc:330] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:57.074780 32678 raft_consensus.cc:2229] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:57.075343 32691 tablet_bootstrap.cc:492] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:57.075394 32705 tablet_bootstrap.cc:492] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:57.075402 32678 pending_rounds.cc:62] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Trying to abort 1 pending ops.
I20260809 14:29:57.075646 32678 pending_rounds.cc:68] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626623115534336 op_type: NO_OP noop_request { }
I20260809 14:29:57.076054 32678 raft_consensus.cc:2866] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:57.084008 32306 tablet_service.cc:1514] Processing DeleteTablet for tablet c48a1586a9164629a7dfe057bbf12dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:57.085421 32763 ts_tablet_manager.cc:1905] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:57.087440 32677 tablet_replica.cc:330] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:57.088065 32677 raft_consensus.cc:2229] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:57.088445 32677 raft_consensus.cc:2258] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:57.076346 32678 raft_consensus.cc:2258] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:57.089965 32677 ts_tablet_manager.cc:1905] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:57.094091 32678 ts_tablet_manager.cc:1905] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:57.104175 32705 tablet_bootstrap.cc:654] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.111760 32677 ts_tablet_manager.cc:1918] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:57.112159 32677 log.cc:1196] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/c48a1586a9164629a7dfe057bbf12dc0
I20260809 14:29:57.113224 32677 ts_tablet_manager.cc:1939] T c48a1586a9164629a7dfe057bbf12dc0 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:57.122319 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]) successfully deleted
I20260809 14:29:57.129870 32705 tablet_bootstrap.cc:492] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:57.130311 32705 ts_tablet_manager.cc:1397] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.055s user 0.014s sys 0.000s
I20260809 14:29:57.133930 32678 ts_tablet_manager.cc:1918] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:57.134366 32678 log.cc:1196] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/c48a1586a9164629a7dfe057bbf12dc0
I20260809 14:29:57.135068 32678 ts_tablet_manager.cc:1939] T c48a1586a9164629a7dfe057bbf12dc0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:57.138015 32691 tablet_bootstrap.cc:654] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.140955 32705 raft_consensus.cc:348] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.141672 32705 raft_consensus.cc:374] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.141566 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5680c3dd85d14f089c9ef93afaf56372" } new_table_name: "default.Table_5289aef101164079880ed9574dcc7209" modify_external_catalogs: true
I20260809 14:29:57.141919 32705 raft_consensus.cc:729] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.141762 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c56b0856d2cf42319b8f01d7f1ab871b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:57.144187 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c56b0856d2cf42319b8f01d7f1ab871b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:57.157894 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9edb9781dce8451b8d8fba51d2fc0394"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:57.160254 32705 consensus_queue.cc:260] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:57.161722 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9edb9781dce8451b8d8fba51d2fc0394 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:57.162381 32705 ts_tablet_manager.cc:1428] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260809 14:29:57.163062 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]) successfully deleted
I20260809 14:29:57.163398 32705 tablet_bootstrap.cc:492] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:57.167110 32691 tablet_bootstrap.cc:492] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:57.167510 32691 ts_tablet_manager.cc:1397] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.092s user 0.007s sys 0.002s
I20260809 14:29:57.178320 32691 raft_consensus.cc:348] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.179103 32691 raft_consensus.cc:374] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.179353 32691 raft_consensus.cc:729] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:29:57.179979 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 consensus state has no leader
I20260809 14:29:57.182089 32763 ts_tablet_manager.cc:1918] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:57.184223 32763 log.cc:1196] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/c48a1586a9164629a7dfe057bbf12dc0
I20260809 14:29:57.186084 32763 ts_tablet_manager.cc:1939] T c48a1586a9164629a7dfe057bbf12dc0 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:57.222419 32705 tablet_bootstrap.cc:654] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.228386 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_dde9e91054fa449aa8bd2d63b8783e9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:57.241487 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 27e2d8ea00d247f69c14e04728519f8b (DEFAULT_TABLE table=default.Table_acb53b3bc5b24b989ccf44b01709c500 [id=4c072795da8241a18c26f3a01af4dda6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.242892 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e2d8ea00d247f69c14e04728519f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.246265 32691 consensus_queue.cc:260] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.247413 32655 raft_consensus.cc:482] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:57.248087 32655 raft_consensus.cc:504] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.253485 32693 tablet_bootstrap.cc:492] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:57.253974 32693 ts_tablet_manager.cc:1397] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.212s user 0.008s sys 0.013s
I20260809 14:29:57.255656 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c468382a28934d50babf7bbc9a834c08" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:57.256772 32333 raft_consensus.cc:2454] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:57.256747 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b5379daa1a3847ba8a537feff3e38ac8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:57.257381 32693 raft_consensus.cc:348] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.258397 32693 raft_consensus.cc:374] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.258760 32693 raft_consensus.cc:729] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.258886 32443 leader_election.cc:302] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
W20260809 14:29:57.259835 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5379daa1a3847ba8a537feff3e38ac8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:57.259652 32693 consensus_queue.cc:260] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.260367 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet c48a1586a9164629a7dfe057bbf12dc0 (table default.Table_440a4f8d04c24bf682668ec6d4d12a4d [id=6fac5819a44f4152bfe254bf83ef692d]) successfully deleted
I20260809 14:29:57.263291 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c468382a28934d50babf7bbc9a834c08" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:57.263837 32693 ts_tablet_manager.cc:1428] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
W20260809 14:29:57.265013 32439 leader_election.cc:341] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:57.266315 32693 tablet_bootstrap.cc:492] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:57.266304 32655 leader_election.cc:288] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:57.267709 32655 raft_consensus.cc:2781] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:57.268762 32655 raft_consensus.cc:482] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:57.269538 32655 raft_consensus.cc:3037] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:57.282542 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 27e2d8ea00d247f69c14e04728519f8b (DEFAULT_TABLE table=default.Table_acb53b3bc5b24b989ccf44b01709c500 [id=4c072795da8241a18c26f3a01af4dda6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.284035 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e2d8ea00d247f69c14e04728519f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.284377 31939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_acb53b3bc5b24b989ccf44b01709c500" } modify_external_catalogs: true
I20260809 14:29:57.284312 32306 tablet_service.cc:1467] Processing CreateTablet for tablet 27e2d8ea00d247f69c14e04728519f8b (DEFAULT_TABLE table=default.Table_acb53b3bc5b24b989ccf44b01709c500 [id=4c072795da8241a18c26f3a01af4dda6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.285039 31939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_acb53b3bc5b24b989ccf44b01709c500" } modify_external_catalogs: true
I20260809 14:29:57.286082 32306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e2d8ea00d247f69c14e04728519f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.287729 32484 tablet_service.cc:1467] Processing CreateTablet for tablet da7b2e6ca78b4fffb6f8daff1d874afe (DEFAULT_TABLE table=default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.289455 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7b2e6ca78b4fffb6f8daff1d874afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.289887 32140 tablet_service.cc:1467] Processing CreateTablet for tablet da7b2e6ca78b4fffb6f8daff1d874afe (DEFAULT_TABLE table=default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.291492 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7b2e6ca78b4fffb6f8daff1d874afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.293181 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c56b0856d2cf42319b8f01d7f1ab871b" } new_table_name: "default.Table_e7a7c8e0160944e2847c372937ae5d40" modify_external_catalogs: true
I20260809 14:29:57.298442 32309 tablet_service.cc:1467] Processing CreateTablet for tablet da7b2e6ca78b4fffb6f8daff1d874afe (DEFAULT_TABLE table=default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.299691 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7b2e6ca78b4fffb6f8daff1d874afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.300804 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 13f6c6a2ff894ec3a795f0c3f044ec17
I20260809 14:29:57.303587 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 with delete_type TABLET_DATA_DELETED (replaced by tablet e28a765e1854486595683c0ac00da24c) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:57.303957 31942 catalog_manager.cc:6565] ReplaceTablet: tablet 13f6c6a2ff894ec3a795f0c3f044ec17 deleted and replaced by tablet e28a765e1854486595683c0ac00da24c
I20260809 14:29:57.304908 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 with delete_type TABLET_DATA_DELETED (replaced by tablet e28a765e1854486595683c0ac00da24c) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:57.305612 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 with delete_type TABLET_DATA_DELETED (replaced by tablet e28a765e1854486595683c0ac00da24c) from {username='slave'} at 127.0.0.1:44626
W20260809 14:29:57.307762 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13f6c6a2ff894ec3a795f0c3f044ec17 already in progress: creating tablet
W20260809 14:29:57.310201 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13f6c6a2ff894ec3a795f0c3f044ec17 already in progress: creating tablet
W20260809 14:29:57.232112 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dde9e91054fa449aa8bd2d63b8783e9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:29:57.310585 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13f6c6a2ff894ec3a795f0c3f044ec17 already in progress: creating tablet
I20260809 14:29:57.317207 32693 tablet_bootstrap.cc:654] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
W20260809 14:29:57.331928 31945 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 26) took 778 ms (client timeout 1000 ms). Trace:
I20260809 14:29:57.333902 32139 tablet_service.cc:1467] Processing CreateTablet for tablet ce085227492e49549d3304a05a66a27d (DEFAULT_TABLE table=default.Table_7b47ee94db13435cb8e6ea1c46c528f7 [id=1240bfe259ae4a2596755c9b3d3d7474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.335652 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce085227492e49549d3304a05a66a27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.337009 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 765d102c8c6444b89da084f27a4cf8ed
I20260809 14:29:57.341639 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 765d102c8c6444b89da084f27a4cf8ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:57.342746 32677 tablet_replica.cc:330] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:57.343521 32677 raft_consensus.cc:2229] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:57.344069 32677 raft_consensus.cc:2258] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:57.344213 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 765d102c8c6444b89da084f27a4cf8ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:57.346277 32677 ts_tablet_manager.cc:1905] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:57.347185 32678 tablet_replica.cc:330] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:57.347322 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet 765d102c8c6444b89da084f27a4cf8ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:57.347952 32678 raft_consensus.cc:2229] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:57.348819 32678 pending_rounds.cc:62] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Trying to abort 1 pending ops.
I20260809 14:29:57.349179 32678 pending_rounds.cc:68] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626622640803840 op_type: NO_OP noop_request { }
I20260809 14:29:57.349692 32678 raft_consensus.cc:2866] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:57.350148 32678 raft_consensus.cc:2258] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:57.353230 32678 ts_tablet_manager.cc:1905] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:57.358085 32763 tablet_replica.cc:330] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
W20260809 14:29:57.334144 31945 rpcz_store.cc:269] 0809 14:29:56.553869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:56.569032 (+ 15163us) service_pool.cc:227] Handling call
0809 14:29:56.579528 (+ 10496us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:29:56.634688 (+ 55160us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4000001378. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa443f8f11 00007ffa4457c851 00007ffa4457c702 00007ffa4d413b09 00007ffa4d413aa0 00007ffa4d54f3b2 00007ffa4d3d30b0 00007ffa4d4d84fe 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71
0809 14:29:57.331823 (+697135us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:29:56.647539 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:29:56.648012 (+ 473us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:29:56.648046 (+ 34us) write_op.cc:435] Acquired schema lock
0809 14:29:56.648067 (+ 21us) tablet.cc:660] Decoding operations
0809 14:29:56.648400 (+ 333us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:29:56.649638 (+ 1238us) write_op.cc:641] Partition lock acquired for write op
0809 14:29:56.649664 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:29:56.649790 (+ 126us) tablet.cc:699] Row locks acquired
0809 14:29:56.649803 (+ 13us) write_op.cc:260] PREPARE: finished
0809 14:29:56.649895 (+ 92us) write_op.cc:270] Start()
0809 14:29:56.649966 (+ 71us) write_op.cc:276] Timestamp: P: 1786285796649842 usec, L: 0
0809 14:29:56.649981 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 14:29:56.650389 (+ 408us) log.cc:842] Serialized 532 byte log entry
0809 14:29:56.922282 (+271893us) op_driver.cc:444] REPLICATION: finished
0809 14:29:56.924427 (+ 2145us) write_op.cc:301] APPLY: starting
0809 14:29:56.924545 (+ 118us) tablet.cc:1365] starting BulkCheckPresence
0809 14:29:56.924693 (+ 148us) tablet.cc:1368] finished BulkCheckPresence
0809 14:29:56.924837 (+ 144us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:29:56.949632 (+ 24795us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:29:56.949992 (+ 360us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:29:56.950087 (+ 95us) write_op.cc:312] APPLY: finished
0809 14:29:56.950337 (+ 250us) log.cc:842] Serialized 22 byte log entry
0809 14:29:56.950645 (+ 308us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:29:56.950789 (+ 144us) write_op.cc:454] Released schema lock
0809 14:29:56.950879 (+ 90us) write_op.cc:341] FINISH: Updating metrics
0809 14:29:56.951220 (+ 341us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:29:56.951268 (+ 48us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":51269376,"child_traces":[["op",{"apply.queue_time_us":1949,"apply.run_cpu_time_us":6200,"apply.run_wall_time_us":27161,"mutex_wait_us":4695,"num_ops":1,"prepare.queue_time_us":9224,"prepare.run_cpu_time_us":1946,"prepare.run_wall_time_us":3298,"raft.queue_time_us":83,"raft.run_cpu_time_us":21,"raft.run_wall_time_us":21,"replication_time_us":272225}]]}
I20260809 14:29:57.374948 32491 tablet_service.cc:1467] Processing CreateTablet for tablet ce085227492e49549d3304a05a66a27d (DEFAULT_TABLE table=default.Table_7b47ee94db13435cb8e6ea1c46c528f7 [id=1240bfe259ae4a2596755c9b3d3d7474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.377231 32308 tablet_service.cc:1467] Processing CreateTablet for tablet e378d7763c5046249011a6d6e0617c97 (DEFAULT_TABLE table=default.Table_3ecd77174e9948a59988f0b186dc8793 [id=82601d705044402cbf2820f18668da15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.378343 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce085227492e49549d3304a05a66a27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.379776 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e378d7763c5046249011a6d6e0617c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.372941 32307 tablet_service.cc:1467] Processing CreateTablet for tablet ce085227492e49549d3304a05a66a27d (DEFAULT_TABLE table=default.Table_7b47ee94db13435cb8e6ea1c46c528f7 [id=1240bfe259ae4a2596755c9b3d3d7474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.381728 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce085227492e49549d3304a05a66a27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.382326 32490 tablet_service.cc:1467] Processing CreateTablet for tablet e378d7763c5046249011a6d6e0617c97 (DEFAULT_TABLE table=default.Table_3ecd77174e9948a59988f0b186dc8793 [id=82601d705044402cbf2820f18668da15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.384281 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e378d7763c5046249011a6d6e0617c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.393342 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c0bf6a05de414660a2f0c1ea7b5a83ef" } new_table_name: "default.Table_d200d3ada57d40339da8cd5f8499cabc" modify_external_catalogs: true
I20260809 14:29:57.407016 32143 tablet_service.cc:1467] Processing CreateTablet for tablet e378d7763c5046249011a6d6e0617c97 (DEFAULT_TABLE table=default.Table_3ecd77174e9948a59988f0b186dc8793 [id=82601d705044402cbf2820f18668da15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.416596 32763 raft_consensus.cc:2229] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:57.417862 32763 raft_consensus.cc:2258] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:57.314016 31938 catalog_manager.cc:5495] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:29:57.422111 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:29:57.433848 32763 ts_tablet_manager.cc:1905] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:29:57.436920 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 178188us
I20260809 14:29:57.441934 32655 raft_consensus.cc:504] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.445976 32691 ts_tablet_manager.cc:1428] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.278s user 0.010s sys 0.002s
I20260809 14:29:57.447198 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_44c7814ca8f941818b618a348ebdc0aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:57.449483 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44c7814ca8f941818b618a348ebdc0aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:57.451567 32691 tablet_bootstrap.cc:492] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:57.439538 32705 tablet_bootstrap.cc:492] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:57.452793 32705 ts_tablet_manager.cc:1397] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.290s user 0.004s sys 0.016s
I20260809 14:29:57.455112 32705 raft_consensus.cc:348] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.455724 32705 raft_consensus.cc:374] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.455971 32705 raft_consensus.cc:729] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.461998 32145 tablet_service.cc:1467] Processing CreateTablet for tablet e846e795144e48e3b2a1eca8c1b26489 (DEFAULT_TABLE table=default.Table_f72f5e4a712b4a519089aa13f2060a2a [id=90072fc6ca834e5a8ab518ccd2131c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.465320 31939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 27e2d8ea00d247f69c14e04728519f8b
I20260809 14:29:57.468406 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e2d8ea00d247f69c14e04728519f8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:57.471400 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e2d8ea00d247f69c14e04728519f8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:57.483094 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e378d7763c5046249011a6d6e0617c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.483310 32485 tablet_service.cc:1467] Processing CreateTablet for tablet e846e795144e48e3b2a1eca8c1b26489 (DEFAULT_TABLE table=default.Table_f72f5e4a712b4a519089aa13f2060a2a [id=90072fc6ca834e5a8ab518ccd2131c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.484387 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_4488bd5aa0734b52a742a371dc2c995b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:57.485491 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e846e795144e48e3b2a1eca8c1b26489. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:57.487035 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4488bd5aa0734b52a742a371dc2c995b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:57.488852 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e846e795144e48e3b2a1eca8c1b26489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.488842 32705 consensus_queue.cc:260] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.493391 32655 leader_election.cc:288] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:57.494836 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c468382a28934d50babf7bbc9a834c08" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:57.495652 32333 raft_consensus.cc:3037] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:57.502745 32333 raft_consensus.cc:2454] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:57.505062 32693 tablet_bootstrap.cc:492] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:57.505574 32691 tablet_bootstrap.cc:654] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.505576 32693 ts_tablet_manager.cc:1397] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.239s user 0.041s sys 0.071s
I20260809 14:29:57.504333 32443 leader_election.cc:302] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:57.509680 32655 raft_consensus.cc:2781] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:57.510497 32655 raft_consensus.cc:686] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:29:57.511780 32655 consensus_queue.cc:237] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.508888 32693 raft_consensus.cc:348] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.515726 32693 raft_consensus.cc:374] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.516075 32693 raft_consensus.cc:729] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.519078 32310 tablet_service.cc:1467] Processing CreateTablet for tablet e846e795144e48e3b2a1eca8c1b26489 (DEFAULT_TABLE table=default.Table_f72f5e4a712b4a519089aa13f2060a2a [id=90072fc6ca834e5a8ab518ccd2131c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:57.521193 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e846e795144e48e3b2a1eca8c1b26489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:57.525846 32693 consensus_queue.cc:260] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.541576 32693 ts_tablet_manager.cc:1428] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.036s user 0.008s sys 0.000s
I20260809 14:29:57.543632 32693 tablet_bootstrap.cc:492] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:57.560748 32705 ts_tablet_manager.cc:1428] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.108s user 0.004s sys 0.002s
I20260809 14:29:57.564878 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c468382a28934d50babf7bbc9a834c08" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:57.568732 32691 tablet_bootstrap.cc:492] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:57.569132 32691 ts_tablet_manager.cc:1397] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.118s user 0.005s sys 0.009s
I20260809 14:29:57.577037 32705 tablet_bootstrap.cc:492] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:57.580506 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b5379daa1a3847ba8a537feff3e38ac8" } modify_external_catalogs: true
I20260809 14:29:57.581547 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b5379daa1a3847ba8a537feff3e38ac8" } modify_external_catalogs: true
W20260809 14:29:57.585613 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 810c39c60b1942e2ab1e48624aeb0ee2 has no consensus state
I20260809 14:29:57.572255 32691 raft_consensus.cc:348] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.586408 32691 raft_consensus.cc:374] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.586695 32691 raft_consensus.cc:729] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.587529 32691 consensus_queue.cc:260] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.591059 301 raft_consensus.cc:482] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:29:57.598325 32439 leader_election.cc:341] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:57.578413 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e2d8ea00d247f69c14e04728519f8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:57.600951 32677 ts_tablet_manager.cc:1918] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:57.601421 32677 log.cc:1196] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/765d102c8c6444b89da084f27a4cf8ed
I20260809 14:29:57.602356 32677 ts_tablet_manager.cc:1939] T 765d102c8c6444b89da084f27a4cf8ed P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:57.605324 318 raft_consensus.cc:482] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:57.606189 318 raft_consensus.cc:504] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.612270 318 leader_election.cc:288] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:57.612862 301 raft_consensus.cc:504] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.614562 32691 ts_tablet_manager.cc:1428] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.045s user 0.004s sys 0.003s
I20260809 14:29:57.615033 301 leader_election.cc:288] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:57.616276 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca78fdbf9ba41c9889903b58898e94d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:57.616896 32333 raft_consensus.cc:2454] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:57.618348 32443 leader_election.cc:302] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:57.619848 318 raft_consensus.cc:2781] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:57.620219 318 raft_consensus.cc:482] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:57.620507 32691 tablet_bootstrap.cc:492] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:57.620595 318 raft_consensus.cc:3037] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:29:57.630391 31930 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260809 14:29:57.631309 32691 tablet_bootstrap.cc:654] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.638722 32678 ts_tablet_manager.cc:1918] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:57.639165 32678 log.cc:1196] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/765d102c8c6444b89da084f27a4cf8ed
I20260809 14:29:57.639957 32678 ts_tablet_manager.cc:1939] T 765d102c8c6444b89da084f27a4cf8ed P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:57.567203 312 raft_consensus.cc:482] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:57.663944 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]) successfully deleted
I20260809 14:29:57.669598 318 raft_consensus.cc:504] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:57.675141 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 27e2d8ea00d247f69c14e04728519f8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27e2d8ea00d247f69c14e04728519f8b already in progress: creating tablet
I20260809 14:29:57.676330 318 leader_election.cc:288] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:57.689468 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]) successfully deleted
I20260809 14:29:57.691705 32691 tablet_bootstrap.cc:492] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:57.692339 32691 ts_tablet_manager.cc:1397] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.072s user 0.008s sys 0.008s
I20260809 14:29:57.700815 32691 raft_consensus.cc:348] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.702870 32691 raft_consensus.cc:374] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.703276 32691 raft_consensus.cc:729] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.705737 32693 tablet_bootstrap.cc:654] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.706352 32691 consensus_queue.cc:260] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.705257 301 raft_consensus.cc:482] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:57.714232 301 raft_consensus.cc:504] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.717855 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca78fdbf9ba41c9889903b58898e94d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:57.719023 32333 raft_consensus.cc:3037] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:57.722283 31946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3ecd77174e9948a59988f0b186dc8793" } modify_external_catalogs: true
I20260809 14:29:57.723311 31946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3ecd77174e9948a59988f0b186dc8793" } modify_external_catalogs: true
I20260809 14:29:57.723874 301 leader_election.cc:288] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:29:57.724886 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 27e2d8ea00d247f69c14e04728519f8b because the tablet was not found. No further retry: Not found: Tablet not found: 27e2d8ea00d247f69c14e04728519f8b
I20260809 14:29:57.727207 32691 ts_tablet_manager.cc:1428] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.034s user 0.007s sys 0.000s
I20260809 14:29:57.736159 318 raft_consensus.cc:482] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:29:57.737316 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 422067edd08745bebdf59d4da704e665 has no consensus state
I20260809 14:29:57.736974 318 raft_consensus.cc:504] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.745577 318 leader_election.cc:288] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:57.756696 32691 tablet_bootstrap.cc:492] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:57.758496 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_a711da642c4642b883663fca660d7be8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:57.760746 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a711da642c4642b883663fca660d7be8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:57.766461 32693 tablet_bootstrap.cc:492] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:57.767033 32693 ts_tablet_manager.cc:1397] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.224s user 0.013s sys 0.002s
I20260809 14:29:57.770258 32691 tablet_bootstrap.cc:654] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.793697 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_4adc37cd84f84f3fb4f7347c01a0e498" } new_table_name: "default.Table_259db63a3b974d09ae70ab777de2033f" modify_external_catalogs: true
I20260809 14:29:57.826280 32691 tablet_bootstrap.cc:492] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:57.826679 32691 ts_tablet_manager.cc:1397] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.072s user 0.021s sys 0.026s
I20260809 14:29:57.830605 32691 raft_consensus.cc:348] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.831238 32691 raft_consensus.cc:374] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.831460 32691 raft_consensus.cc:729] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.831992 32691 consensus_queue.cc:260] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.672509 312 raft_consensus.cc:504] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.834882 32691 ts_tablet_manager.cc:1428] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260809 14:29:57.835282 312 leader_election.cc:288] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:57.835937 32691 tablet_bootstrap.cc:492] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:57.853852 330 raft_consensus.cc:482] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:57.854761 330 raft_consensus.cc:504] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.836030 312 raft_consensus.cc:482] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:57.863045 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_ba8cab0449354ab3aa7efe707df20ab3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:57.875728 32691 tablet_bootstrap.cc:654] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:29:57.879904 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba8cab0449354ab3aa7efe707df20ab3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:57.896761 32691 tablet_bootstrap.cc:492] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:57.897185 32691 ts_tablet_manager.cc:1397] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.014s
I20260809 14:29:57.899559 32691 raft_consensus.cc:348] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.903152 330 leader_election.cc:288] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:57.900282 32691 raft_consensus.cc:374] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.908165 32691 raft_consensus.cc:729] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.910630 32691 consensus_queue.cc:260] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.895270 312 raft_consensus.cc:504] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:57.924209 32691 ts_tablet_manager.cc:1428] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260809 14:29:57.925329 32691 tablet_bootstrap.cc:492] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:57.928030 312 leader_election.cc:288] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:57.941311 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca78fdbf9ba41c9889903b58898e94d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:57.941885 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca78fdbf9ba41c9889903b58898e94d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:57.942806 32170 raft_consensus.cc:2454] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:57.944272 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969849d30c204937a7508e8d54dae427" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:57.944566 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049678aadd4549d1b831174727647ec8" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:57.945938 32515 raft_consensus.cc:2454] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:57.947877 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca78fdbf9ba41c9889903b58898e94d" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:57.949139 32514 raft_consensus.cc:2379] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
W20260809 14:29:57.949726 32439 leader_election.cc:341] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:57.958972 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e873f3b715c424e8693dbcba0990fad" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:57.960841 32514 raft_consensus.cc:2379] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:29:57.967073 32691 tablet_bootstrap.cc:654] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.994851 32095 leader_election.cc:302] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:57.995349 32691 tablet_bootstrap.cc:492] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:57.995777 32691 ts_tablet_manager.cc:1397] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.071s user 0.008s sys 0.008s
I20260809 14:29:57.997816 32705 tablet_bootstrap.cc:654] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:57.998047 32691 raft_consensus.cc:348] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:57.998737 32691 raft_consensus.cc:374] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:57.999034 32691 raft_consensus.cc:729] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:57.999787 32691 consensus_queue.cc:260] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.009709 312 raft_consensus.cc:2781] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:58.010195 312 raft_consensus.cc:482] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:58.010571 312 raft_consensus.cc:3037] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.013190 325 raft_consensus.cc:482] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:58.014015 325 raft_consensus.cc:504] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.021066 32691 ts_tablet_manager.cc:1428] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.025s user 0.004s sys 0.003s
I20260809 14:29:58.024753 32691 tablet_bootstrap.cc:492] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:29:58.029927 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (table default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 13f6c6a2ff894ec3a795f0c3f044ec17 consensus state has no leader
I20260809 14:29:58.042626 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca78fdbf9ba41c9889903b58898e94d" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:58.043013 32691 tablet_bootstrap.cc:654] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.043323 32170 raft_consensus.cc:2454] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:29:58.046114 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca78fdbf9ba41c9889903b58898e94d" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:58.047477 32514 raft_consensus.cc:2379] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3eb0bc287b12421d89abf97ca66e4d9b in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
I20260809 14:29:58.052403 32705 tablet_bootstrap.cc:492] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:58.052946 32705 ts_tablet_manager.cc:1397] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.476s user 0.006s sys 0.010s
I20260809 14:29:58.055624 32705 raft_consensus.cc:348] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.056280 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969849d30c204937a7508e8d54dae427" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:58.057407 32334 raft_consensus.cc:2454] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:58.058962 32443 leader_election.cc:302] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:58.060009 32260 leader_election.cc:302] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:29:58.060252 318 raft_consensus.cc:2781] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:58.060590 318 raft_consensus.cc:482] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:58.060901 318 raft_consensus.cc:3037] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.063416 32693 raft_consensus.cc:348] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.064260 32693 raft_consensus.cc:374] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.064549 32693 raft_consensus.cc:729] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.065145 32693 consensus_queue.cc:260] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.056469 32705 raft_consensus.cc:374] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.067276 32705 raft_consensus.cc:729] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.072853 32705 consensus_queue.cc:260] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.073915 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849d798165144531810a03dc2b9b5f1e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:58.074540 32170 raft_consensus.cc:2454] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:29:58.078153 32705 ts_tablet_manager.cc:1428] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.025s user 0.006s sys 0.001s
I20260809 14:29:58.084571 32705 tablet_bootstrap.cc:492] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:58.093072 32170 raft_consensus.cc:3037] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.097558 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849d798165144531810a03dc2b9b5f1e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:58.098505 32514 raft_consensus.cc:2454] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:29:58.108127 312 raft_consensus.cc:504] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.110009 312 leader_election.cc:288] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.110800 32333 raft_consensus.cc:2454] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:58.111955 318 raft_consensus.cc:504] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.112998 32443 leader_election.cc:302] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:58.113838 318 leader_election.cc:288] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:29:58.114313 318 raft_consensus.cc:2781] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:58.114729 318 raft_consensus.cc:686] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:29:58.114825 32705 tablet_bootstrap.cc:654] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.115360 301 raft_consensus.cc:2741] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 14:29:58.115947 32763 ts_tablet_manager.cc:1918] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:58.116326 32763 log.cc:1196] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/765d102c8c6444b89da084f27a4cf8ed
I20260809 14:29:58.117002 32763 ts_tablet_manager.cc:1939] T 765d102c8c6444b89da084f27a4cf8ed P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:58.122253 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969849d30c204937a7508e8d54dae427" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:29:58.123569 32439 leader_election.cc:341] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:58.123883 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049678aadd4549d1b831174727647ec8" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
W20260809 14:29:58.124801 32306 consensus_meta.cc:287] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.630s user 0.007s sys 0.004s
I20260809 14:29:58.125887 32514 raft_consensus.cc:3037] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.127449 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9edb9781dce8451b8d8fba51d2fc0394" } new_table_name: "default.Table_ffe722b702a44148abf3d980648b96e4" modify_external_catalogs: true
I20260809 14:29:58.123777 318 consensus_queue.cc:237] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.140650 32705 tablet_bootstrap.cc:492] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:58.141149 32705 ts_tablet_manager.cc:1397] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.057s user 0.011s sys 0.000s
I20260809 14:29:58.143816 32705 raft_consensus.cc:348] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.144649 32705 raft_consensus.cc:374] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.144961 32705 raft_consensus.cc:729] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.147332 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca78fdbf9ba41c9889903b58898e94d" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:58.148176 32333 raft_consensus.cc:2379] T dca78fdbf9ba41c9889903b58898e94d P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
I20260809 14:29:58.150458 32693 ts_tablet_manager.cc:1428] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.383s user 0.006s sys 0.006s
I20260809 14:29:58.151633 32693 tablet_bootstrap.cc:492] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:58.151724 32260 leader_election.cc:302] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:58.159385 32705 consensus_queue.cc:260] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:58.167075 32260 consensus_peers.cc:487] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:29:58.169845 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849d798165144531810a03dc2b9b5f1e" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:58.170604 32515 raft_consensus.cc:2454] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
W20260809 14:29:58.171059 31938 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 36) took 930 ms (client timeout 1000 ms). Trace:
I20260809 14:29:58.171939 32095 leader_election.cc:302] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:58.164197 31940 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:29:58.175168 325 leader_election.cc:288] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:29:58.179136 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 66240.8us
I20260809 14:29:58.184098 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e873f3b715c424e8693dbcba0990fad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:58.185099 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet 765d102c8c6444b89da084f27a4cf8ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:58.189105 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 969849d30c204937a7508e8d54dae427 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:58.196257 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet 049678aadd4549d1b831174727647ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet ce085227492e49549d3304a05a66a27d) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:58.198729 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet 5951ffedd0cd481f9f425f0af3b8ad01 with delete_type TABLET_DATA_DELETED (replaced by tablet 609b3e6049ec4b0582d78037b7416710) from {username='slave'} at 127.0.0.1:41544
W20260809 14:29:58.201798 32099 leader_election.cc:341] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 049678aadd4549d1b831174727647ec8
I20260809 14:29:58.202363 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e873f3b715c424e8693dbcba0990fad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:29:58.204870 31940 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 38) took 859 ms (client timeout 1000 ms). Trace:
I20260809 14:29:58.211491 32514 raft_consensus.cc:2454] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
W20260809 14:29:58.220255 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 48412.8us
I20260809 14:29:58.221490 32095 leader_election.cc:302] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:29:58.222496 330 raft_consensus.cc:2781] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:58.223001 330 raft_consensus.cc:686] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:29:58.224260 32763 tablet_replica.cc:330] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:29:58.223915 330 consensus_queue.cc:237] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.226560 325 raft_consensus.cc:2781] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:58.226881 325 raft_consensus.cc:482] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:58.227301 325 raft_consensus.cc:3037] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.232779 32099 leader_election.cc:302] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:29:58.234349 312 raft_consensus.cc:3037] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.168100 301 raft_consensus.cc:2781] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:58.173643 338 consensus_queue.cc:1035] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:29:58.175989 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e873f3b715c424e8693dbcba0990fad" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:58.177310 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849d798165144531810a03dc2b9b5f1e" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:58.180897 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969849d30c204937a7508e8d54dae427" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:58.189845 32693 tablet_bootstrap.cc:654] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.240828 32331 raft_consensus.cc:3037] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:29:58.171722 31938 rpcz_store.cc:269] 0809 14:29:57.240298 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:57.244729 (+ 4431us) service_pool.cc:227] Handling call
0809 14:29:58.170993 (+926264us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:29:57.652743 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:29:57.653111 (+ 368us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:29:57.653131 (+ 20us) write_op.cc:435] Acquired schema lock
0809 14:29:57.653142 (+ 11us) tablet.cc:660] Decoding operations
0809 14:29:57.653435 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:29:57.653471 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0809 14:29:57.653494 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:29:57.653652 (+ 158us) tablet.cc:699] Row locks acquired
0809 14:29:57.653666 (+ 14us) write_op.cc:260] PREPARE: finished
0809 14:29:57.653777 (+ 111us) write_op.cc:270] Start()
0809 14:29:57.653862 (+ 85us) write_op.cc:276] Timestamp: P: 1786285797653726 usec, L: 0
0809 14:29:57.653890 (+ 28us) op_driver.cc:331] REPLICATION: starting
0809 14:29:57.654444 (+ 554us) log.cc:842] Serialized 532 byte log entry
0809 14:29:58.040600 (+386156us) op_driver.cc:444] REPLICATION: finished
0809 14:29:58.106700 (+ 66100us) write_op.cc:301] APPLY: starting
0809 14:29:58.106753 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0809 14:29:58.106841 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0809 14:29:58.106932 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:29:58.107076 (+ 144us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:29:58.107191 (+ 115us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:29:58.107234 (+ 43us) write_op.cc:312] APPLY: finished
0809 14:29:58.160567 (+ 53333us) log.cc:842] Serialized 22 byte log entry
0809 14:29:58.161217 (+ 650us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:29:58.161277 (+ 60us) write_op.cc:454] Released schema lock
0809 14:29:58.161308 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0809 14:29:58.162167 (+ 859us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:29:58.162192 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":55917,"apply.run_cpu_time_us":1465,"apply.run_wall_time_us":55838,"num_ops":1,"prepare.queue_time_us":191901,"prepare.run_cpu_time_us":2003,"prepare.run_wall_time_us":36348,"raft.queue_time_us":80,"raft.run_cpu_time_us":18,"raft.run_wall_time_us":18,"replication_time_us":386637,"thread_start_us":74443,"threads_started":1}]]}
I20260809 14:29:58.246330 32334 raft_consensus.cc:2454] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:29:58.247996 301 raft_consensus.cc:482] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:58.192314 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet a9f61933c2004a43af2c748ea445942c from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:58.252328 301 raft_consensus.cc:3037] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.225445 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 765d102c8c6444b89da084f27a4cf8ed (table default.Table_9ecd2a39ee0f40cca949c529cba934ac [id=5cfe6abc061c4d39a8da29bc24541eb2]) successfully deleted
W20260809 14:29:58.254581 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 27e2d8ea00d247f69c14e04728519f8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27e2d8ea00d247f69c14e04728519f8b already in progress: creating tablet
W20260809 14:29:58.245258 31940 rpcz_store.cc:269] 0809 14:29:57.346017 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:57.357053 (+ 11036us) service_pool.cc:227] Handling call
0809 14:29:58.204776 (+847723us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":4}
W20260809 14:29:58.253670 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 87158.4us
I20260809 14:29:58.238620 339 consensus_queue.cc:1035] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260809 14:29:58.267104 32705 ts_tablet_manager.cc:1428] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.126s user 0.008s sys 0.000s
I20260809 14:29:58.268369 32705 tablet_bootstrap.cc:492] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:58.270110 344 tablet_replica.cc:330] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:58.271617 344 raft_consensus.cc:2229] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:58.278620 32693 tablet_bootstrap.cc:492] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:58.279049 32693 ts_tablet_manager.cc:1397] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.128s user 0.016s sys 0.005s
I20260809 14:29:58.278789 341 consensus_queue.cc:1035] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 14:29:58.281526 32099 leader_election.cc:341] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 049678aadd4549d1b831174727647ec8
I20260809 14:29:58.291407 32693 raft_consensus.cc:348] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.292171 32693 raft_consensus.cc:374] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.295110 32691 tablet_bootstrap.cc:492] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:58.296875 32691 ts_tablet_manager.cc:1397] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.272s user 0.051s sys 0.029s
I20260809 14:29:58.297736 32693 raft_consensus.cc:729] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.298379 32693 consensus_queue.cc:260] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.300058 32691 raft_consensus.cc:348] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.301672 339 consensus_queue.cc:1035] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:29:58.306525 32691 raft_consensus.cc:374] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.307559 32691 raft_consensus.cc:729] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.308230 32693 ts_tablet_manager.cc:1428] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260809 14:29:58.309890 32691 consensus_queue.cc:260] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.311792 32693 tablet_bootstrap.cc:492] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
W20260809 14:29:58.312919 32260 consensus_peers.cc:487] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Couldn't send request to peer 6bed14aff53a41b2bbf5a13ce55dec5a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:29:58.317044 32331 raft_consensus.cc:2454] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:58.330400 32443 leader_election.cc:302] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:58.332576 318 raft_consensus.cc:2781] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:58.333106 318 raft_consensus.cc:686] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:29:58.334017 318 consensus_queue.cc:237] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.336956 32693 tablet_bootstrap.cc:654] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.342851 31946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet e378d7763c5046249011a6d6e0617c97
I20260809 14:29:58.343896 31943 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 99d1ccab7f4c4e55b93fcbced822039d
I20260809 14:29:58.346009 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet e378d7763c5046249011a6d6e0617c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:58.330358 32691 ts_tablet_manager.cc:1428] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260809 14:29:58.349630 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e873f3b715c424e8693dbcba0990fad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:58.350033 32691 tablet_bootstrap.cc:492] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:58.355296 32693 tablet_bootstrap.cc:492] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:58.355765 32693 ts_tablet_manager.cc:1397] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.044s user 0.010s sys 0.000s
I20260809 14:29:58.358500 32693 raft_consensus.cc:348] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.359566 32693 raft_consensus.cc:374] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.365590 32306 tablet_service.cc:1514] Processing DeleteTablet for tablet e378d7763c5046249011a6d6e0617c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:58.368547 32693 raft_consensus.cc:729] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.369357 32693 consensus_queue.cc:260] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.370561 32691 tablet_bootstrap.cc:654] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:29:58.371001 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 104189us
I20260809 14:29:58.373663 32763 raft_consensus.cc:2229] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:58.377174 312 raft_consensus.cc:2726] T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:58.378093 312 raft_consensus.cc:482] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:58.378840 312 raft_consensus.cc:504] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.380290 32693 ts_tablet_manager.cc:1428] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260809 14:29:58.386132 32693 tablet_bootstrap.cc:492] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:58.388916 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet e378d7763c5046249011a6d6e0617c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:58.397148 32099 leader_election.cc:302] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:29:58.420805 32693 tablet_bootstrap.cc:654] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.422101 32763 pending_rounds.cc:62] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:29:58.420136 301 raft_consensus.cc:504] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.422611 32763 pending_rounds.cc:68] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626629541085184 op_type: NO_OP noop_request { }
I20260809 14:29:58.423663 32763 raft_consensus.cc:2866] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:58.424984 301 leader_election.cc:288] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.426308 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849d798165144531810a03dc2b9b5f1e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:58.441123 312 leader_election.cc:288] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.424683 32763 raft_consensus.cc:2258] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:58.453555 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c468382a28934d50babf7bbc9a834c08" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:58.454523 32693 tablet_bootstrap.cc:492] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:58.454962 32693 ts_tablet_manager.cc:1397] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.069s user 0.007s sys 0.009s
I20260809 14:29:58.457880 301 raft_consensus.cc:482] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:58.458426 301 raft_consensus.cc:504] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.461572 301 leader_election.cc:288] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.463887 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c8eb8ad2dd406085779ed6a86eb70e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:58.458457 32693 raft_consensus.cc:348] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.464777 32693 raft_consensus.cc:374] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.465076 32693 raft_consensus.cc:729] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.465719 32693 consensus_queue.cc:260] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:29:58.466154 32260 leader_election.cc:341] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:58.469256 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849d798165144531810a03dc2b9b5f1e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:58.471782 32513 raft_consensus.cc:3037] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.474252 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c8eb8ad2dd406085779ed6a86eb70e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:58.475183 32693 ts_tablet_manager.cc:1428] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.020s user 0.003s sys 0.006s
I20260809 14:29:58.474939 32514 raft_consensus.cc:2454] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:29:58.476320 32693 tablet_bootstrap.cc:492] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:58.478757 32260 leader_election.cc:302] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:29:58.485991 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f36ab4503f24ca2a2a0d812c4c52468 has no consensus state
I20260809 14:29:58.486637 301 raft_consensus.cc:2781] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:58.487030 301 raft_consensus.cc:482] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:58.487552 301 raft_consensus.cc:3037] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.497758 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_6d9b31cc5d9d4976b94613e05d76120c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:58.502076 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d9b31cc5d9d4976b94613e05d76120c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:58.506880 32691 tablet_bootstrap.cc:492] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:58.507306 32691 ts_tablet_manager.cc:1397] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.157s user 0.006s sys 0.002s
I20260809 14:29:58.470131 344 raft_consensus.cc:2258] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:58.511176 32691 raft_consensus.cc:348] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.512041 32691 raft_consensus.cc:374] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.512391 32691 raft_consensus.cc:729] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.511847 344 ts_tablet_manager.cc:1905] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:58.536720 32693 tablet_bootstrap.cc:654] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.553336 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e7a7c8e0160944e2847c372937ae5d40" } modify_external_catalogs: true
I20260809 14:29:58.513149 32691 consensus_queue.cc:260] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.558014 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e7a7c8e0160944e2847c372937ae5d40" } modify_external_catalogs: true
I20260809 14:29:58.559725 32691 ts_tablet_manager.cc:1428] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.052s user 0.005s sys 0.005s
I20260809 14:29:58.524154 32513 raft_consensus.cc:2454] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:29:58.563207 31948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_259db63a3b974d09ae70ab777de2033f" } modify_external_catalogs: true
I20260809 14:29:58.563753 31948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_259db63a3b974d09ae70ab777de2033f" } modify_external_catalogs: true
I20260809 14:29:58.565423 32260 leader_election.cc:302] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:58.565541 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5f55e858e8a946809527fffb7b5a04dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:58.569157 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f55e858e8a946809527fffb7b5a04dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:58.575249 32691 tablet_bootstrap.cc:492] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:58.585750 32691 tablet_bootstrap.cc:654] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.588317 339 raft_consensus.cc:2781] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:58.589684 353 tablet_replica.cc:330] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:58.589933 339 raft_consensus.cc:686] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:29:58.595304 339 consensus_queue.cc:237] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:58.597725 347 log.cc:924] Time spent T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b: Append to log took a long time: real 0.153s user 0.000s sys 0.001s
I20260809 14:29:58.614704 301 raft_consensus.cc:504] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:58.613349 345 consensus_queue.cc:1159] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260809 14:29:58.621335 353 raft_consensus.cc:2229] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:29:58.622918 353 pending_rounds.cc:62] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 1 pending ops.
I20260809 14:29:58.623310 353 pending_rounds.cc:68] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626623838031872 op_type: NO_OP noop_request { }
I20260809 14:29:58.623989 353 raft_consensus.cc:2866] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:29:58.624583 353 raft_consensus.cc:2258] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:58.618590 325 raft_consensus.cc:504] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:29:58.583257 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb799e344ae544f9b4df659462bddba2 has no consensus state
I20260809 14:29:58.608605 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet a9f61933c2004a43af2c748ea445942c
I20260809 14:29:58.640414 31945 catalog_manager.cc:6565] ReplaceTablet: tablet a9f61933c2004a43af2c748ea445942c deleted and replaced by tablet 2dc8d0cad33b4778af9fe9247743c38f
I20260809 14:29:58.644562 32169 raft_consensus.cc:3037] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.648274 330 raft_consensus.cc:2726] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:58.652959 32168 raft_consensus.cc:2379] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3eb0bc287b12421d89abf97ca66e4d9b in current term 1: Already voted for candidate e1f94e0d792c46c9a0b6ab42820da6eb in this term.
I20260809 14:29:58.656666 32763 ts_tablet_manager.cc:1905] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:58.663452 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f61933c2004a43af2c748ea445942c with delete_type TABLET_DATA_DELETED (replaced by tablet 2dc8d0cad33b4778af9fe9247743c38f) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:58.668815 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c8eb8ad2dd406085779ed6a86eb70e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:58.672097 301 leader_election.cc:288] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.675001 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c8eb8ad2dd406085779ed6a86eb70e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:58.675742 32513 raft_consensus.cc:3037] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:58.689881 325 leader_election.cc:288] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.692652 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849d798165144531810a03dc2b9b5f1e" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:58.693360 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f61933c2004a43af2c748ea445942c with delete_type TABLET_DATA_DELETED (replaced by tablet 2dc8d0cad33b4778af9fe9247743c38f) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:58.693902 32514 raft_consensus.cc:2379] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:29:58.695746 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f61933c2004a43af2c748ea445942c with delete_type TABLET_DATA_DELETED (replaced by tablet 2dc8d0cad33b4778af9fe9247743c38f) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:58.699956 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_44c7814ca8f941818b618a348ebdc0aa" } new_table_name: "default.Table_487d358ef97f4504ac4d4d31e491ea25" modify_external_catalogs: true
I20260809 14:29:58.702849 32513 raft_consensus.cc:2454] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
W20260809 14:29:58.708668 32260 leader_election.cc:341] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:58.710031 32260 leader_election.cc:302] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:58.714691 32705 tablet_bootstrap.cc:654] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.717675 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_710d9b1281b94c84a497faea0c441199"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:58.723592 318 raft_consensus.cc:482] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:58.721650 32691 tablet_bootstrap.cc:492] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:58.724481 318 raft_consensus.cc:504] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.725975 32691 ts_tablet_manager.cc:1397] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.151s user 0.030s sys 0.014s
I20260809 14:29:58.730703 318 leader_election.cc:288] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:58.730589 32691 raft_consensus.cc:348] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.731549 32691 raft_consensus.cc:374] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.731865 32691 raft_consensus.cc:729] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.738648 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f6c6a2ff894ec3a795f0c3f044ec17" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:58.740100 32334 raft_consensus.cc:2454] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:58.750905 32691 consensus_queue.cc:260] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.753626 32691 ts_tablet_manager.cc:1428] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.027s user 0.005s sys 0.003s
I20260809 14:29:58.754261 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_bb02de675fd247e5af1ff74a6c57f2bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:58.758432 32443 leader_election.cc:302] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:58.748167 301 raft_consensus.cc:2781] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:29:58.761132 301 raft_consensus.cc:686] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:29:58.762719 301 consensus_queue.cc:237] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.750241 353 ts_tablet_manager.cc:1905] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:58.772650 318 raft_consensus.cc:2781] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:58.773296 318 raft_consensus.cc:482] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:58.775167 318 raft_consensus.cc:3037] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:29:58.715422 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 343180us
I20260809 14:29:58.786197 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c468382a28934d50babf7bbc9a834c08" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:58.728668 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f6c6a2ff894ec3a795f0c3f044ec17" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:29:58.796273 32439 leader_election.cc:341] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:58.803484 325 raft_consensus.cc:482] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:58.804756 325 raft_consensus.cc:504] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.806663 325 leader_election.cc:288] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.807387 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849d798165144531810a03dc2b9b5f1e" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:58.807906 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658ab8e11b37419ba2176a9175bfd936" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:58.810467 339 raft_consensus.cc:482] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:58.811251 339 raft_consensus.cc:504] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:29:58.811625 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb02de675fd247e5af1ff74a6c57f2bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:58.814863 339 leader_election.cc:288] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.818045 341 consensus_queue.cc:1035] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:29:58.818394 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f61933c2004a43af2c748ea445942c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:58.819281 32513 raft_consensus.cc:2454] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:29:58.821048 32260 leader_election.cc:302] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:58.822415 339 raft_consensus.cc:2781] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260809 14:29:58.822322 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_710d9b1281b94c84a497faea0c441199 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:58.822949 339 raft_consensus.cc:482] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:58.823482 339 raft_consensus.cc:3037] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:29:58.835971 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b53f7d398d64701b4504db5c9c440b8 has no consensus state
I20260809 14:29:58.841357 32691 tablet_bootstrap.cc:492] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:58.846299 32099 leader_election.cc:302] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:29:58.846400 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_331f2b568f3b4937b41ddb58af580e97" } new_table_name: "default.Table_8792541442324807881089a25803c23e" modify_external_catalogs: true
I20260809 14:29:58.847394 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658ab8e11b37419ba2176a9175bfd936" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:58.849944 32691 tablet_bootstrap.cc:654] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:58.866653 32693 tablet_bootstrap.cc:492] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:58.867139 32693 ts_tablet_manager.cc:1397] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.391s user 0.006s sys 0.019s
I20260809 14:29:58.871675 32693 raft_consensus.cc:348] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:29:58.883512 31930 consensus_peers.cc:487] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 -> Peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Couldn't send request to peer 9d10512ac8b3406098fad1f7da012ab3. Status: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260809 14:29:58.894119 318 raft_consensus.cc:504] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.895556 32693 raft_consensus.cc:374] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:58.895978 32693 raft_consensus.cc:729] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:29:58.896845 32693 consensus_queue.cc:260] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.850107 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f61933c2004a43af2c748ea445942c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:58.902176 338 raft_consensus.cc:482] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:58.908816 338 raft_consensus.cc:504] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:58.917821 338 leader_election.cc:288] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:58.924481 32693 ts_tablet_manager.cc:1428] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.057s user 0.002s sys 0.007s
I20260809 14:29:58.928164 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aad93fcc8b42e6b55db6b3ed6b4f91" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:58.932124 325 raft_consensus.cc:2726] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:29:58.933727 32260 leader_election.cc:341] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:58.937861 32693 tablet_bootstrap.cc:492] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:58.898938 341 raft_consensus.cc:482] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:58.944972 341 raft_consensus.cc:504] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.957039 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f6c6a2ff894ec3a795f0c3f044ec17" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:58.958534 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f6c6a2ff894ec3a795f0c3f044ec17" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:58.959434 32334 raft_consensus.cc:3037] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:29:58.961005 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 114811us
W20260809 14:29:58.964442 32260 leader_election.cc:341] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:58.965720 32099 leader_election.cc:302] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:29:58.972172 325 raft_consensus.cc:2726] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 14:29:58.972604 318 leader_election.cc:288] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:58.973820 32099 leader_election.cc:302] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:29:58.982766 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c consensus state has no leader
I20260809 14:29:58.997104 32705 tablet_bootstrap.cc:492] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:29:58.997859 32705 ts_tablet_manager.cc:1397] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.730s user 0.010s sys 0.012s
I20260809 14:29:59.000730 325 raft_consensus.cc:2726] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:59.015588 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:58.985966 32693 tablet_bootstrap.cc:654] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
W20260809 14:29:59.021236 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 56870.9us
I20260809 14:29:58.997592 32334 raft_consensus.cc:2454] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:59.000620 353 ts_tablet_manager.cc:1918] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:59.029192 353 log.cc:1196] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/6e873f3b715c424e8693dbcba0990fad
I20260809 14:29:59.031610 353 ts_tablet_manager.cc:1939] T 6e873f3b715c424e8693dbcba0990fad P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:29:59.016992 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9ae593e06b4e2a949b799380c1a3e3" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:29:59.033521 32331 raft_consensus.cc:2454] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:29:59.034706 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5e0ce7fb17174f28bf6eed6e3f36193e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:59.049692 32705 raft_consensus.cc:348] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:59.050539 32705 raft_consensus.cc:374] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:59.052724 32705 raft_consensus.cc:729] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:29:59.050558 353 tablet_replica.cc:330] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:29:59.065061 32705 consensus_queue.cc:260] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:59.066140 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_7d6f721ab1324625bcf5ac42e3e20f8d" } new_table_name: "default.Table_305490483ff34e8bbb83440bf4b0ae50" modify_external_catalogs: true
I20260809 14:29:58.985529 32691 tablet_bootstrap.cc:492] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:59.071903 339 raft_consensus.cc:504] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:58.993052 341 leader_election.cc:288] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:59.075786 32693 tablet_bootstrap.cc:492] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:29:59.076963 32693 ts_tablet_manager.cc:1397] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.139s user 0.013s sys 0.012s
I20260809 14:29:59.077287 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_8693ee189dfe49b4b713d3291505bdfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:29:59.047741 32443 leader_election.cc:302] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:59.080582 353 raft_consensus.cc:2229] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:59.080987 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f61933c2004a43af2c748ea445942c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:59.082245 32443 leader_election.cc:302] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:29:59.083871 376 raft_consensus.cc:2781] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
W20260809 14:29:59.084007 32260 leader_election.cc:341] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:59.084800 318 raft_consensus.cc:482] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:59.086625 353 raft_consensus.cc:2258] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:59.085424 318 raft_consensus.cc:504] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:59.091385 318 leader_election.cc:288] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:59.092406 32693 raft_consensus.cc:348] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.042691 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 6e873f3b715c424e8693dbcba0990fad (table default.Table_733ab1fa87b647d2961d60a43900c952 [id=0d4ec30e1ef44803b533141ba0e57fec]) successfully deleted
I20260809 14:29:59.093292 32693 raft_consensus.cc:374] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 14:29:59.069876 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 116717us
W20260809 14:29:59.093446 31933 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 50790.6us
I20260809 14:29:59.093714 32693 raft_consensus.cc:729] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:29:59.095499 31933 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c (table default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c consensus state has no leader
W20260809 14:29:59.097170 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8693ee189dfe49b4b713d3291505bdfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.097216 32693 consensus_queue.cc:260] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.098142 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9ae593e06b4e2a949b799380c1a3e3" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:29:59.096092 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet e378d7763c5046249011a6d6e0617c97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e378d7763c5046249011a6d6e0617c97 already in progress: creating tablet
W20260809 14:29:59.103221 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e0ce7fb17174f28bf6eed6e3f36193e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.105604 32693 ts_tablet_manager.cc:1428] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.028s user 0.004s sys 0.004s
I20260809 14:29:59.106355 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aad93fcc8b42e6b55db6b3ed6b4f91" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:29:59.100663 376 raft_consensus.cc:686] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:29:59.107301 32513 raft_consensus.cc:2454] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:29:59.109758 32693 tablet_bootstrap.cc:492] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:29:59.108805 376 consensus_queue.cc:237] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.112485 375 raft_consensus.cc:2781] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:59.112954 375 raft_consensus.cc:482] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:59.113654 375 raft_consensus.cc:3037] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:59.123467 32691 ts_tablet_manager.cc:1397] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.282s user 0.014s sys 0.007s
I20260809 14:29:59.124251 32260 leader_election.cc:302] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:59.125506 338 raft_consensus.cc:2781] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:29:59.126086 338 raft_consensus.cc:482] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:29:59.126688 338 raft_consensus.cc:3037] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:59.129195 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aad93fcc8b42e6b55db6b3ed6b4f91" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:29:59.128597 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 46519.3us
I20260809 14:29:59.136324 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aad93fcc8b42e6b55db6b3ed6b4f91" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:29:59.140010 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dc8d0cad33b4778af9fe9247743c38f has no consensus state
W20260809 14:29:59.141726 32439 leader_election.cc:341] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:29:59.143224 32439 leader_election.cc:341] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:59.148269 344 ts_tablet_manager.cc:1918] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:59.148959 344 log.cc:1196] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/6e873f3b715c424e8693dbcba0990fad
I20260809 14:29:59.150033 344 ts_tablet_manager.cc:1939] T 6e873f3b715c424e8693dbcba0990fad P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:59.155623 318 consensus_queue.cc:1035] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:29:59.160053 32691 raft_consensus.cc:348] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.160718 32691 raft_consensus.cc:374] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:59.160946 32691 raft_consensus.cc:729] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:59.161489 32691 consensus_queue.cc:260] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:29:59.163036 32439 leader_election.cc:341] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:59.163843 32691 ts_tablet_manager.cc:1428] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.040s user 0.003s sys 0.001s
I20260809 14:29:59.172289 318 consensus_queue.cc:1035] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:29:59.176807 339 leader_election.cc:288] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:59.184273 338 raft_consensus.cc:504] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:59.188699 353 ts_tablet_manager.cc:1905] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:59.189349 32334 raft_consensus.cc:2379] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6bed14aff53a41b2bbf5a13ce55dec5a in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:29:59.184787 32693 tablet_bootstrap.cc:654] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:59.199633 338 leader_election.cc:288] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:29:59.208561 32691 tablet_bootstrap.cc:492] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:59.208353 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f924e07646a94fd7a82006cef63a8abe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:59.211131 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f924e07646a94fd7a82006cef63a8abe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.209030 32443 leader_election.cc:302] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:59.214862 318 raft_consensus.cc:3037] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:59.210017 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f61933c2004a43af2c748ea445942c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:59.186707 375 raft_consensus.cc:504] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.222949 32513 raft_consensus.cc:3037] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:59.222817 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_4be51cdd7c0f4caa9b2ba3719af75258"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:59.226991 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4be51cdd7c0f4caa9b2ba3719af75258 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.210602 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aad93fcc8b42e6b55db6b3ed6b4f91" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:29:59.229202 344 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 55) took 1035 ms. Trace:
I20260809 14:29:59.232635 344 rpcz_store.cc:276] 0809 14:29:58.183605 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.183899 (+ 294us) service_pool.cc:227] Handling call
0809 14:29:58.269868 (+ 85969us) ts_tablet_manager.cc:1145] Deleting tablet 6e873f3b715c424e8693dbcba0990fad
0809 14:29:58.269937 (+ 69us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:29:58.797930 (+527993us) tablet_metadata.cc:641] Metadata flushed
0809 14:29:59.147893 (+349963us) tablet_metadata.cc:641] Metadata flushed
0809 14:29:59.218658 (+ 70765us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":85054,"thread_start_us":85722,"threads_started":1}
I20260809 14:29:59.237535 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 6e873f3b715c424e8693dbcba0990fad (table default.Table_733ab1fa87b647d2961d60a43900c952 [id=0d4ec30e1ef44803b533141ba0e57fec]) successfully deleted
I20260809 14:29:59.239760 375 leader_election.cc:288] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:59.239815 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aad93fcc8b42e6b55db6b3ed6b4f91" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:59.240897 344 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 56) took 1056 ms. Trace:
I20260809 14:29:59.242646 344 rpcz_store.cc:276] 0809 14:29:58.184756 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.184936 (+ 180us) service_pool.cc:227] Handling call
0809 14:29:59.237948 (+1053012us) ts_tablet_manager.cc:1145] Deleting tablet 765d102c8c6444b89da084f27a4cf8ed
0809 14:29:59.240245 (+ 2297us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:29:59.240779 (+ 534us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1052292}
I20260809 14:29:59.244302 32705 ts_tablet_manager.cc:1428] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.246s user 0.009s sys 0.001s
W20260809 14:29:59.244336 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 765d102c8c6444b89da084f27a4cf8ed because the tablet was not found. No further retry: Not found: Tablet not found: 765d102c8c6444b89da084f27a4cf8ed
I20260809 14:29:59.267554 32763 ts_tablet_manager.cc:1918] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:29:59.268378 32763 log.cc:1196] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/049678aadd4549d1b831174727647ec8
I20260809 14:29:59.269848 32763 ts_tablet_manager.cc:1939] T 049678aadd4549d1b831174727647ec8 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:29:59.274537 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 (table default.Table_5289aef101164079880ed9574dcc7209 [id=032f6264a2c842f1ba6b007331ccfcc6]). Expected version 4 got 0
I20260809 14:29:59.273655 32705 tablet_bootstrap.cc:492] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:29:59.289939 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 3705cf965d214eb794984b9c16604326 (DEFAULT_TABLE table=default.Table_f7fd4627a85c4f358f737fad5077415c [id=12165d071bb546219c54224908e47280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.276117 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9ae593e06b4e2a949b799380c1a3e3" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:29:59.278049 32143 tablet_service.cc:1467] Processing CreateTablet for tablet cb799e344ae544f9b4df659462bddba2 (DEFAULT_TABLE table=default.Table_ffe722b702a44148abf3d980648b96e4 [id=6866e71fe868487cb904cab993a14371]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.280570 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 422067edd08745bebdf59d4da704e665 (DEFAULT_TABLE table=default.Table_d200d3ada57d40339da8cd5f8499cabc [id=8d157c6db35a4a6f976fed683f2d1d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.294418 31937 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8693ee189dfe49b4b713d3291505bdfe" } new_table_name: "default.Table_abda07ff24e547bfb6075ac285c136dd" modify_external_catalogs: true
I20260809 14:29:59.248980 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9ae593e06b4e2a949b799380c1a3e3" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:29:59.295756 32334 raft_consensus.cc:3037] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:59.301353 32513 raft_consensus.cc:2454] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
W20260809 14:29:59.261808 32260 leader_election.cc:341] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:59.306452 32485 tablet_service.cc:1467] Processing CreateTablet for tablet 3705cf965d214eb794984b9c16604326 (DEFAULT_TABLE table=default.Table_f7fd4627a85c4f358f737fad5077415c [id=12165d071bb546219c54224908e47280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.307436 32491 tablet_service.cc:1467] Processing CreateTablet for tablet cb799e344ae544f9b4df659462bddba2 (DEFAULT_TABLE table=default.Table_ffe722b702a44148abf3d980648b96e4 [id=6866e71fe868487cb904cab993a14371]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.306305 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 422067edd08745bebdf59d4da704e665 (DEFAULT_TABLE table=default.Table_d200d3ada57d40339da8cd5f8499cabc [id=8d157c6db35a4a6f976fed683f2d1d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.308331 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3705cf965d214eb794984b9c16604326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.308773 32310 tablet_service.cc:1467] Processing CreateTablet for tablet cb799e344ae544f9b4df659462bddba2 (DEFAULT_TABLE table=default.Table_ffe722b702a44148abf3d980648b96e4 [id=6866e71fe868487cb904cab993a14371]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.309759 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb799e344ae544f9b4df659462bddba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.311122 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb799e344ae544f9b4df659462bddba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.312584 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 422067edd08745bebdf59d4da704e665. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:59.312379 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 48214.7us
I20260809 14:29:59.315315 32306 tablet_service.cc:1467] Processing CreateTablet for tablet 422067edd08745bebdf59d4da704e665 (DEFAULT_TABLE table=default.Table_d200d3ada57d40339da8cd5f8499cabc [id=8d157c6db35a4a6f976fed683f2d1d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.305351 32260 leader_election.cc:302] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:29:59.317168 32306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 422067edd08745bebdf59d4da704e665. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:59.317317 32260 raft_consensus.cc:2670] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:29:59.303431 318 raft_consensus.cc:2726] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:29:59.283859 31946 catalog_manager.cc:5495] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 1 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } }
I20260809 14:29:59.337471 31940 master_service.cc:946] ReplaceTablet: received request to replace tablet 6e873f3b715c424e8693dbcba0990fad from {username='slave'} at 127.0.0.1:49782
I20260809 14:29:59.341951 376 consensus_queue.cc:1035] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:29:59.322755 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:29:59.343007 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 (DEFAULT_TABLE table=default.Table_259db63a3b974d09ae70ab777de2033f [id=e4e69ddcbdf647cb9e6fd48d47afca31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.346519 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f36ab4503f24ca2a2a0d812c4c52468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.349866 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 3705cf965d214eb794984b9c16604326 (DEFAULT_TABLE table=default.Table_f7fd4627a85c4f358f737fad5077415c [id=12165d071bb546219c54224908e47280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.351940 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3705cf965d214eb794984b9c16604326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.356758 301 raft_consensus.cc:482] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:59.357478 301 raft_consensus.cc:504] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.359398 32334 raft_consensus.cc:2454] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:29:59.368512 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 (DEFAULT_TABLE table=default.Table_259db63a3b974d09ae70ab777de2033f [id=e4e69ddcbdf647cb9e6fd48d47afca31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.370460 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f36ab4503f24ca2a2a0d812c4c52468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.373185 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 5ff78e5cb7eb4796a8afa4847f0d2834 (DEFAULT_TABLE table=default.Table_5352320f0eb24ca5a85ceb658929954a [id=0cc322f8067843d98204a74e6016db47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.374686 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ff78e5cb7eb4796a8afa4847f0d2834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.375542 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 (DEFAULT_TABLE table=default.Table_259db63a3b974d09ae70ab777de2033f [id=e4e69ddcbdf647cb9e6fd48d47afca31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.379004 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 5ff78e5cb7eb4796a8afa4847f0d2834 (DEFAULT_TABLE table=default.Table_5352320f0eb24ca5a85ceb658929954a [id=0cc322f8067843d98204a74e6016db47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.363675 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3705cf965d214eb794984b9c16604326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.387188 32691 tablet_bootstrap.cc:654] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:59.383764 32305 tablet_service.cc:1467] Processing CreateTablet for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c (DEFAULT_TABLE table=default.Table_f3f904c0b94440ddb1f6f7f733689d8c [id=ae6b2507dcdd4367b1faa278e84fbe3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.393617 32305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.392362 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5f55e858e8a946809527fffb7b5a04dc" } new_table_name: "default.Table_aaad0560a0364aad9a26bbbcecc64736" modify_external_catalogs: true
W20260809 14:29:59.412326 32439 leader_election.cc:341] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:59.414996 32304 tablet_service.cc:1467] Processing CreateTablet for tablet e28a765e1854486595683c0ac00da24c (DEFAULT_TABLE table=default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.415274 32514 raft_consensus.cc:2454] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:29:59.427906 31948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 6f36ab4503f24ca2a2a0d812c4c52468
I20260809 14:29:59.428920 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 5ff78e5cb7eb4796a8afa4847f0d2834 (DEFAULT_TABLE table=default.Table_5352320f0eb24ca5a85ceb658929954a [id=0cc322f8067843d98204a74e6016db47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.430557 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_8b32f662e5154cf78a133dd5249f5f0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:59.432390 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b32f662e5154cf78a133dd5249f5f0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.436019 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c (DEFAULT_TABLE table=default.Table_f3f904c0b94440ddb1f6f7f733689d8c [id=ae6b2507dcdd4367b1faa278e84fbe3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.438994 32489 tablet_service.cc:1467] Processing CreateTablet for tablet e28a765e1854486595683c0ac00da24c (DEFAULT_TABLE table=default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.443411 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 (DEFAULT_TABLE table=default.Table_e7a7c8e0160944e2847c372937ae5d40 [id=f2ed48a0b8934893874b5fe8d5c6617d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.444234 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 810c39c60b1942e2ab1e48624aeb0ee2
I20260809 14:29:59.449353 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28a765e1854486595683c0ac00da24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.451517 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ff78e5cb7eb4796a8afa4847f0d2834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.453455 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.457305 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28a765e1854486595683c0ac00da24c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:59.461592 31937 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 07ac3ef5573c40b894c9b2e743ca0582 has no consensus state
I20260809 14:29:59.466194 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 810c39c60b1942e2ab1e48624aeb0ee2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:59.486240 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5121e9f5a85a45d285d78f41a442769a has no consensus state
I20260809 14:29:59.494493 318 raft_consensus.cc:482] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:59.495020 318 raft_consensus.cc:504] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.497284 318 leader_election.cc:288] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:29:59.471558 344 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 57) took 1126 ms. Trace:
I20260809 14:29:59.499130 344 rpcz_store.cc:276] 0809 14:29:58.345647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.345806 (+ 159us) service_pool.cc:227] Handling call
0809 14:29:59.426752 (+1080946us) ts_tablet_manager.cc:1145] Deleting tablet e378d7763c5046249011a6d6e0617c97
0809 14:29:59.428145 (+ 1393us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:29:59.469780 (+ 41635us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1074169}
I20260809 14:29:59.500331 344 tablet_replica.cc:330] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:59.501196 344 raft_consensus.cc:2229] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 14:29:59.522996 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet e378d7763c5046249011a6d6e0617c97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e378d7763c5046249011a6d6e0617c97 already in progress: creating tablet
I20260809 14:29:59.525319 32141 tablet_service.cc:1467] Processing CreateTablet for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c (DEFAULT_TABLE table=default.Table_f3f904c0b94440ddb1f6f7f733689d8c [id=ae6b2507dcdd4367b1faa278e84fbe3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.505537 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 609b3e6049ec4b0582d78037b7416710 (DEFAULT_TABLE table=default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.533908 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609b3e6049ec4b0582d78037b7416710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.540860 344 raft_consensus.cc:2258] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:59.543911 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_828cdc150d4e475ca9247f5dba9c8e25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:59.546953 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_828cdc150d4e475ca9247f5dba9c8e25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.552686 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:59.564807 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:59.564108 32691 tablet_bootstrap.cc:492] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:59.587955 32691 ts_tablet_manager.cc:1397] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.383s user 0.010s sys 0.009s
I20260809 14:29:59.580986 31938 catalog_manager.cc:5495] T dca78fdbf9ba41c9889903b58898e94d P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 1 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } }
I20260809 14:29:59.605309 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_13a67d50c6c746e99d3c0ce47b81f6ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:59.607070 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13a67d50c6c746e99d3c0ce47b81f6ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.617079 31938 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:29:59.621235 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 609b3e6049ec4b0582d78037b7416710 (DEFAULT_TABLE table=default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.622021 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb799e344ae544f9b4df659462bddba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.624061 32136 tablet_service.cc:1514] Processing DeleteTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:59.602241 338 raft_consensus.cc:482] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:59.623113 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:59.629223 32146 tablet_service.cc:1467] Processing CreateTablet for tablet e28a765e1854486595683c0ac00da24c (DEFAULT_TABLE table=default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:29:59.631682 31946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 37) took 1194 ms (client timeout 1000 ms). Trace:
I20260809 14:29:59.637221 31937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_7b47ee94db13435cb8e6ea1c46c528f7" } modify_external_catalogs: true
I20260809 14:29:59.637857 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 (DEFAULT_TABLE table=default.Table_e7a7c8e0160944e2847c372937ae5d40 [id=f2ed48a0b8934893874b5fe8d5c6617d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.638408 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 71aad93fcc8b42e6b55db6b3ed6b4f91
I20260809 14:29:59.638350 31937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_7b47ee94db13435cb8e6ea1c46c528f7" } modify_external_catalogs: true
I20260809 14:29:59.641304 31945 catalog_manager.cc:6565] ReplaceTablet: tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 deleted and replaced by tablet 2f171260cb764ac284aef85e4219767e
I20260809 14:29:59.643323 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.649425 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ff78e5cb7eb4796a8afa4847f0d2834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.652011 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_828cdc150d4e475ca9247f5dba9c8e25" } modify_external_catalogs: true
I20260809 14:29:59.653136 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f36ab4503f24ca2a2a0d812c4c52468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.653113 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_828cdc150d4e475ca9247f5dba9c8e25" } modify_external_catalogs: true
I20260809 14:29:59.544167 344 ts_tablet_manager.cc:1905] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:59.656783 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 422067edd08745bebdf59d4da704e665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.678084 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f171260cb764ac284aef85e4219767e) from {username='slave'} at 127.0.0.1:41544
I20260809 14:29:59.679885 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28a765e1854486595683c0ac00da24c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:29:59.637225 31946 rpcz_store.cc:269] 0809 14:29:58.437552 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.437733 (+ 181us) service_pool.cc:227] Handling call
0809 14:29:58.491663 (+ 53930us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:29:59.631502 (+1139839us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:29:59.523374 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:29:59.523727 (+ 353us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:29:59.523750 (+ 23us) write_op.cc:435] Acquired schema lock
0809 14:29:59.523762 (+ 12us) tablet.cc:660] Decoding operations
0809 14:29:59.524005 (+ 243us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:29:59.524039 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0809 14:29:59.524059 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:29:59.524186 (+ 127us) tablet.cc:699] Row locks acquired
0809 14:29:59.524198 (+ 12us) write_op.cc:260] PREPARE: finished
0809 14:29:59.524295 (+ 97us) write_op.cc:270] Start()
0809 14:29:59.524370 (+ 75us) write_op.cc:276] Timestamp: P: 1786285799524260 usec, L: 0
0809 14:29:59.524385 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 14:29:59.524804 (+ 419us) log.cc:842] Serialized 533 byte log entry
0809 14:29:59.562186 (+ 37382us) op_driver.cc:444] REPLICATION: finished
0809 14:29:59.563041 (+ 855us) write_op.cc:301] APPLY: starting
0809 14:29:59.563109 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0809 14:29:59.563278 (+ 169us) tablet.cc:1368] finished BulkCheckPresence
0809 14:29:59.563526 (+ 248us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:29:59.563747 (+ 221us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:29:59.563982 (+ 235us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:29:59.564066 (+ 84us) write_op.cc:312] APPLY: finished
0809 14:29:59.564320 (+ 254us) log.cc:842] Serialized 23 byte log entry
0809 14:29:59.564526 (+ 206us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:29:59.564627 (+ 101us) write_op.cc:454] Released schema lock
0809 14:29:59.564677 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0809 14:29:59.564911 (+ 234us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:29:59.564996 (+ 85us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":114708736,"child_traces":[["op",{"apply.queue_time_us":646,"apply.run_cpu_time_us":2137,"apply.run_wall_time_us":2136,"mutex_wait_us":21911,"num_ops":1,"prepare.queue_time_us":132111,"prepare.run_cpu_time_us":1666,"prepare.run_wall_time_us":2832,"raft.queue_time_us":125,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":11,"replication_time_us":37686}]]}
I20260809 14:29:59.687218 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e1f46b3673cb4694925d4248e8ef6c81" } new_table_name: "default.Table_be3acab0711d448d95c162af38ba7180" modify_external_catalogs: true
I20260809 14:29:59.692628 31946 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_abda07ff24e547bfb6075ac285c136dd" } new_table_name: "default.Table_abacee8a40094093b6e3454fedfa8ca6" modify_external_catalogs: true
I20260809 14:29:59.692981 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609b3e6049ec4b0582d78037b7416710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.643723 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f171260cb764ac284aef85e4219767e) from {username='slave'} at 127.0.0.1:44626
I20260809 14:29:59.592432 32691 raft_consensus.cc:348] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.717496 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 810c39c60b1942e2ab1e48624aeb0ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:29:59.720850 32691 raft_consensus.cc:374] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:59.721155 32691 raft_consensus.cc:729] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:59.721961 32691 consensus_queue.cc:260] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.723502 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:29:59.739212 32169 raft_consensus.cc:3037] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:29:59.760339 32691 ts_tablet_manager.cc:1428] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.172s user 0.004s sys 0.004s
I20260809 14:29:59.761240 32691 tablet_bootstrap.cc:492] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:59.780453 392 raft_consensus.cc:482] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:29:59.781127 392 raft_consensus.cc:504] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:29:59.786803 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f332ca6d630349c9a3d8561b63890931"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:29:59.790391 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f332ca6d630349c9a3d8561b63890931 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.799496 392 leader_election.cc:288] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:29:59.802551 382 log.cc:924] Time spent T dca78fdbf9ba41c9889903b58898e94d P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.175s user 0.000s sys 0.000s
I20260809 14:29:59.818005 32691 tablet_bootstrap.cc:654] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:59.288174 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 54) took 1092 ms. Trace:
I20260809 14:29:59.833077 32763 rpcz_store.cc:276] 0809 14:29:58.195357 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.195670 (+ 313us) service_pool.cc:227] Handling call
0809 14:29:58.224109 (+ 28439us) ts_tablet_manager.cc:1145] Deleting tablet 049678aadd4549d1b831174727647ec8
0809 14:29:58.224145 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:29:58.421156 (+197011us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54000b0ed4. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181471 00007fb6c7204352 00007fb6c761a26c 00007fb6c8227cfa 00007fb6c8257e10 00007fb6c823ee0d 00007fb6c823edb9 00007fb6c823ed49 00007fb6c823ed11 00007fb6c823df3d 0000000000580784
0809 14:29:58.867858 (+446702us) tablet_metadata.cc:641] Metadata flushed
0809 14:29:59.216933 (+349075us) tablet_metadata.cc:641] Metadata flushed
0809 14:29:59.288088 (+ 71155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":338733824,"tablet-delete.queue_time_us":25636}
I20260809 14:29:59.835261 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 049678aadd4549d1b831174727647ec8 (table default.Table_7b47ee94db13435cb8e6ea1c46c528f7 [id=1240bfe259ae4a2596755c9b3d3d7474]) successfully deleted
I20260809 14:29:59.838588 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:59.606400 32302 tablet_service.cc:1467] Processing CreateTablet for tablet 609b3e6049ec4b0582d78037b7416710 (DEFAULT_TABLE table=default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:29:59.836447 344 ts_tablet_manager.cc:1918] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:29:59.858891 344 log.cc:1196] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/a9f61933c2004a43af2c748ea445942c
I20260809 14:29:59.861523 344 ts_tablet_manager.cc:1939] T a9f61933c2004a43af2c748ea445942c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:29:59.866858 344 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 58) took 1171 ms. Trace:
I20260809 14:29:59.856653 32691 tablet_bootstrap.cc:492] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:59.876469 32691 ts_tablet_manager.cc:1397] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.115s user 0.005s sys 0.006s
I20260809 14:29:59.876202 344 rpcz_store.cc:276] 0809 14:29:58.695362 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.695605 (+ 243us) service_pool.cc:227] Handling call
0809 14:29:59.500177 (+804572us) ts_tablet_manager.cc:1145] Deleting tablet a9f61933c2004a43af2c748ea445942c
0809 14:29:59.500227 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:29:59.735561 (+235334us) tablet_metadata.cc:641] Metadata flushed
0809 14:29:59.836126 (+100565us) tablet_metadata.cc:641] Metadata flushed
0809 14:29:59.866787 (+ 30661us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":803931}
I20260809 14:29:59.879025 344 tablet_replica.cc:330] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:29:59.879034 32691 raft_consensus.cc:348] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.881873 32691 raft_consensus.cc:374] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:29:59.882205 32691 raft_consensus.cc:729] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:59.882892 32691 consensus_queue.cc:260] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.885412 32691 ts_tablet_manager.cc:1428] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260809 14:29:59.886335 32691 tablet_bootstrap.cc:492] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:59.891305 344 raft_consensus.cc:2229] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:29:59.891906 344 raft_consensus.cc:2258] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:29:59.896528 32300 tablet_service.cc:1514] Processing DeleteTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:29:59.903805 344 ts_tablet_manager.cc:1905] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:29:59.913950 32691 tablet_bootstrap.cc:654] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:59.929603 32705 tablet_bootstrap.cc:700] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent opening tablet: real 0.173s user 0.001s sys 0.000s
I20260809 14:29:59.930430 32705 tablet_bootstrap.cc:654] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:29:59.932443 32691 tablet_bootstrap.cc:492] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:29:59.933053 32691 ts_tablet_manager.cc:1397] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.047s user 0.010s sys 0.001s
I20260809 14:29:59.935330 32691 raft_consensus.cc:348] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.936048 32691 raft_consensus.cc:374] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 14:29:59.968261 32439 leader_election.cc:341] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:29:59.977921 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet a9f61933c2004a43af2c748ea445942c (table default.Table_305490483ff34e8bbb83440bf4b0ae50 [id=c1935d3bc49c405abe4d006932820224]) successfully deleted
W20260809 14:29:59.978751 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 6f36ab4503f24ca2a2a0d812c4c52468 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f36ab4503f24ca2a2a0d812c4c52468 already in progress: creating tablet
W20260809 14:29:59.979435 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 810c39c60b1942e2ab1e48624aeb0ee2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 810c39c60b1942e2ab1e48624aeb0ee2 already in progress: creating tablet
I20260809 14:29:59.936304 32691 raft_consensus.cc:729] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:29:59.985739 32691 consensus_queue.cc:260] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:29:59.994776 32691 ts_tablet_manager.cc:1428] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.061s user 0.006s sys 0.000s
I20260809 14:30:00.003368 32691 tablet_bootstrap.cc:492] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:00.016022 32763 tablet_replica.cc:330] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:00.016866 32691 tablet_bootstrap.cc:654] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:00.017635 32763 raft_consensus.cc:2229] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:00.018283 32763 raft_consensus.cc:2258] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:00.030247 301 leader_election.cc:288] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:00.033851 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b1f8694507e64827909363f73d0471b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:00.035236 32303 tablet_service.cc:1467] Processing CreateTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 (DEFAULT_TABLE table=default.Table_e7a7c8e0160944e2847c372937ae5d40 [id=f2ed48a0b8934893874b5fe8d5c6617d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:00.035727 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1f8694507e64827909363f73d0471b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:29:59.809924 338 raft_consensus.cc:504] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.042222 32691 tablet_bootstrap.cc:492] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:00.042652 32691 ts_tablet_manager.cc:1397] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.003s
I20260809 14:30:00.043757 32443 leader_election.cc:302] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:00.044839 376 raft_consensus.cc:2781] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:00.045338 376 raft_consensus.cc:686] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:00.045683 32303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 810c39c60b1942e2ab1e48624aeb0ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.046250 376 consensus_queue.cc:237] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.046864 344 ts_tablet_manager.cc:1918] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:00.047400 344 log.cc:1196] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/71aad93fcc8b42e6b55db6b3ed6b4f91
I20260809 14:30:00.052776 344 ts_tablet_manager.cc:1939] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:00.055408 32691 raft_consensus.cc:348] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.056068 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 (table default.Table_5289aef101164079880ed9574dcc7209 [id=032f6264a2c842f1ba6b007331ccfcc6]) successfully deleted
I20260809 14:30:00.057848 32691 raft_consensus.cc:374] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:00.058805 32691 raft_consensus.cc:729] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:00.060176 32691 consensus_queue.cc:260] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.071808 32691 ts_tablet_manager.cc:1428] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.029s user 0.008s sys 0.000s
W20260809 14:30:00.080533 31946 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 07ac3ef5573c40b894c9b2e743ca0582 has no consensus state
I20260809 14:30:00.081521 31946 master_service.cc:946] ReplaceTablet: received request to replace tablet 57c074cef80146e28a3f3795077d047a from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:00.085711 31945 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 4e5a3b39ac3c4c0a8a081c6dc3ea2af6
I20260809 14:30:00.086616 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c1126946af2e49c38aea0adda1c6272a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:00.089648 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1126946af2e49c38aea0adda1c6272a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:00.080294 32691 tablet_bootstrap.cc:492] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:00.093566 32302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609b3e6049ec4b0582d78037b7416710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.098879 32691 tablet_bootstrap.cc:654] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:00.036185 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e873f3b715c424e8693dbcba0990fad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:00.112962 338 leader_election.cc:288] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:00.116503 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f171260cb764ac284aef85e4219767e) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:00.116773 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (table default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 13f6c6a2ff894ec3a795f0c3f044ec17 consensus state has no leader
I20260809 14:30:00.119179 353 ts_tablet_manager.cc:1918] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:00.119549 353 log.cc:1196] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/a9f61933c2004a43af2c748ea445942c
I20260809 14:30:00.120132 32691 tablet_bootstrap.cc:492] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:00.120162 353 ts_tablet_manager.cc:1939] T a9f61933c2004a43af2c748ea445942c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:00.120620 32691 ts_tablet_manager.cc:1397] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.041s user 0.015s sys 0.006s
I20260809 14:30:00.092142 32300 tablet_service.cc:1514] Processing DeleteTablet for tablet 969849d30c204937a7508e8d54dae427 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:00.123020 32691 raft_consensus.cc:348] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.123682 32691 raft_consensus.cc:374] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:00.123914 32691 raft_consensus.cc:729] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:00.127485 32691 consensus_queue.cc:260] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.129915 32691 ts_tablet_manager.cc:1428] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260809 14:30:00.131093 32691 tablet_bootstrap.cc:492] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:29:59.614622 32260 leader_election.cc:302] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:00.134346 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 533959us
I20260809 14:30:00.137086 32691 tablet_bootstrap.cc:654] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:00.136914 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:00.137763 32514 raft_consensus.cc:2454] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:00.141278 338 raft_consensus.cc:2781] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:00.141765 338 raft_consensus.cc:686] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:00.142553 338 consensus_queue.cc:237] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:00.146509 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cfc9e0319748c8b0e928fc64706933" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:00.147320 32513 raft_consensus.cc:2454] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:00.155352 376 raft_consensus.cc:482] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:00.155795 376 raft_consensus.cc:504] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.157744 376 leader_election.cc:288] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:00.165887 32691 tablet_bootstrap.cc:492] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:00.166286 32691 ts_tablet_manager.cc:1397] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.035s user 0.013s sys 0.016s
I20260809 14:30:00.169395 32691 raft_consensus.cc:348] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.170101 32691 raft_consensus.cc:374] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:00.170508 32691 raft_consensus.cc:729] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:00.171211 32691 consensus_queue.cc:260] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.172756 32691 ts_tablet_manager.cc:1428] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260809 14:30:00.174314 32691 tablet_bootstrap.cc:492] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:00.182149 32691 tablet_bootstrap.cc:654] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:00.208626 32306 consensus_meta.cc:287] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.593s user 0.004s sys 0.003s
I20260809 14:30:00.208796 32691 tablet_bootstrap.cc:492] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:00.209156 32691 ts_tablet_manager.cc:1397] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.007s
I20260809 14:30:00.211220 32691 raft_consensus.cc:348] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.211874 32691 raft_consensus.cc:374] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:00.212113 32691 raft_consensus.cc:729] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:00.088308 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969849d30c204937a7508e8d54dae427" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:00.223376 32691 consensus_queue.cc:260] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.224869 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969849d30c204937a7508e8d54dae427" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:00.225832 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb1dd7b6b6243b4ba3eee9b1b3b1f1c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:00.226501 376 consensus_queue.cc:1035] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:00.229019 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:00.237553 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cfc9e0319748c8b0e928fc64706933" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:00.247848 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3c5c49afb5814baf8e9f3f33f5beac33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:00.250020 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c5c49afb5814baf8e9f3f33f5beac33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:00.254386 32169 raft_consensus.cc:3037] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:00.232666 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:00.269054 32439 leader_election.cc:341] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:00.270417 32439 consensus_peers.cc:487] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:00.290918 32763 ts_tablet_manager.cc:1905] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:00.291756 32693 tablet_bootstrap.cc:492] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:00.292244 32693 ts_tablet_manager.cc:1397] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 1.183s user 0.008s sys 0.010s
I20260809 14:30:00.343196 408 raft_consensus.cc:482] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:00.343665 408 raft_consensus.cc:504] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.345865 408 leader_election.cc:288] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:00.346280 408 raft_consensus.cc:482] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:00.346731 408 raft_consensus.cc:504] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.362404 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce085227492e49549d3304a05a66a27d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:00.364135 32143 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 59) took 1095 ms. Trace:
I20260809 14:30:00.364555 32143 rpcz_store.cc:276] 0809 14:29:59.268773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.276498 (+ 7725us) service_pool.cc:227] Handling call
0809 14:29:59.372668 (+ 96170us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:59.372913 (+ 245us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:29:59.921471 (+548558us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.281680 (+360209us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:00.364050 (+ 82370us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 14:30:00.371270 32439 leader_election.cc:341] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:00.376367 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5289aef101164079880ed9574dcc7209" } new_table_name: "default.Table_42864987ccd14c05a6909a6e1527e208" modify_external_catalogs: true
I20260809 14:30:00.385426 318 raft_consensus.cc:482] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:00.386361 318 raft_consensus.cc:504] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.389413 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e2d8ea00d247f69c14e04728519f8b" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:00.392768 32439 leader_election.cc:341] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:00.400096 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_d200d3ada57d40339da8cd5f8499cabc" } modify_external_catalogs: true
I20260809 14:30:00.400887 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_d200d3ada57d40339da8cd5f8499cabc" } modify_external_catalogs: true
I20260809 14:30:00.417814 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3b68d38823a54a27a2654a116d5d0430"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:00.420060 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b68d38823a54a27a2654a116d5d0430 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:00.420843 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 8f35f6dc9a33438db135d9978caf3b14 (DEFAULT_TABLE table=default.Table_f924e07646a94fd7a82006cef63a8abe [id=2d3c076f6ef54647beba4d1564fb9924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.422295 32705 tablet_bootstrap.cc:492] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:00.422873 32705 ts_tablet_manager.cc:1397] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 1.178s user 0.012s sys 0.004s
I20260809 14:30:00.422845 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f35f6dc9a33438db135d9978caf3b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.353711 32332 raft_consensus.cc:2454] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:30:00.429446 32705 raft_consensus.cc:348] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:00.430153 32705 raft_consensus.cc:374] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:00.430438 32705 raft_consensus.cc:729] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:00.431051 32304 tablet_service.cc:1467] Processing CreateTablet for tablet 8f35f6dc9a33438db135d9978caf3b14 (DEFAULT_TABLE table=default.Table_f924e07646a94fd7a82006cef63a8abe [id=2d3c076f6ef54647beba4d1564fb9924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.431720 32483 tablet_service.cc:1467] Processing CreateTablet for tablet ae1af25739774df8a5536c75c2e1f416 (DEFAULT_TABLE table=default.Table_5e0ce7fb17174f28bf6eed6e3f36193e [id=e9c09963654840d38231616ed482bfef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.432184 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f35f6dc9a33438db135d9978caf3b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.433142 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1af25739774df8a5536c75c2e1f416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.436901 32147 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 63) took 1059 ms. Trace:
I20260809 14:30:00.437192 32306 tablet_service.cc:1467] Processing CreateTablet for tablet ae1af25739774df8a5536c75c2e1f416 (DEFAULT_TABLE table=default.Table_5e0ce7fb17174f28bf6eed6e3f36193e [id=e9c09963654840d38231616ed482bfef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.437244 32147 rpcz_store.cc:276] 0809 14:29:59.377327 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.377527 (+ 200us) service_pool.cc:227] Handling call
0809 14:29:59.380010 (+ 2483us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:59.380174 (+ 164us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:29:59.907188 (+527014us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.326815 (+419627us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:00.436837 (+110022us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:00.438251 32306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1af25739774df8a5536c75c2e1f416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.438676 32139 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 61) took 1150 ms. Trace:
I20260809 14:30:00.438983 32139 rpcz_store.cc:276] 0809 14:29:59.288263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.288565 (+ 302us) service_pool.cc:227] Handling call
0809 14:29:59.291072 (+ 2507us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:59.362895 (+ 71823us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:29:59.805892 (+442997us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.351657 (+545765us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:00.438620 (+ 86963us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:00.441112 32310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 57) took 1134 ms. Trace:
I20260809 14:30:00.441431 32310 rpcz_store.cc:276] 0809 14:29:59.306850 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.307167 (+ 317us) service_pool.cc:227] Handling call
0809 14:29:59.310195 (+ 3028us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:59.310369 (+ 174us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:00.125098 (+814729us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.440654 (+315556us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:00.441047 (+ 393us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:00.442736 32331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.30.252.67:59643 (request call id 21) took 1284 ms. Trace:
I20260809 14:30:00.444594 32099 leader_election.cc:302] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:00.443037 32331 rpcz_store.cc:276] 0809 14:29:59.158847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.159108 (+ 261us) service_pool.cc:227] Handling call
0809 14:29:59.159663 (+ 555us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 14:29:59.160159 (+ 496us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 14:29:59.160196 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 14:29:59.162379 (+ 2183us) log.cc:842] Serialized 26 byte log entry
0809 14:30:00.047190 (+884811us) raft_consensus.cc:1631] Marking committed up to 0
0809 14:30:00.085979 (+ 38789us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 14:30:00.105913 (+ 19934us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 14:30:00.330158 (+224245us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b54001093a0. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8398016471 00007f839801f771 00007f839802a220 00007f839802a1c8 00007f839802a148 00007f839802a103 00007f839802938c 00007f839a014206 00007f839a00c808 00007f839806638b 00007f8398068e80
0809 14:30:00.419864 (+ 89706us) raft_consensus.cc:1671] finished
0809 14:30:00.419880 (+ 16us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 14:30:00.442680 (+ 22800us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":661,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":261,"spinlock_wait_cycles":12617984,"thread_start_us":92909,"threads_started":1,"wal-append.queue_time_us":866863}
I20260809 14:30:00.449616 32141 tablet_service.cc:1467] Processing CreateTablet for tablet ae1af25739774df8a5536c75c2e1f416 (DEFAULT_TABLE table=default.Table_5e0ce7fb17174f28bf6eed6e3f36193e [id=e9c09963654840d38231616ed482bfef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.450955 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1af25739774df8a5536c75c2e1f416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.452595 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 56) took 1790 ms. Trace:
I20260809 14:30:00.452960 353 rpcz_store.cc:276] 0809 14:29:58.662520 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.663051 (+ 531us) service_pool.cc:227] Handling call
0809 14:29:59.048846 (+385795us) ts_tablet_manager.cc:1145] Deleting tablet a9f61933c2004a43af2c748ea445942c
0809 14:29:59.049449 (+ 603us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:29:59.086317 (+ 36868us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5400092254. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8398016471 00007f8398099352 00007f83984af26c 00007f83990bccfa 00007f83990ece10 00007f83990d3e0d 00007f83990d3db9 00007f83990d3d49 00007f83990d3d11 00007f83990d2f3d 0000000000580784
0809 14:29:59.290447 (+204130us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.118950 (+828503us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.452516 (+333566us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1306,"spinlock_wait_cycles":41221504,"tablet-delete.queue_time_us":383873}
I20260809 14:30:00.455200 353 tablet_replica.cc:330] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:00.457080 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 (DEFAULT_TABLE table=default.Table_abacee8a40094093b6e3454fedfa8ca6 [id=329d73d7b2244b42805b4b7cd0e2e45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.458223 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ac3ef5573c40b894c9b2e743ca0582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.458576 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 (DEFAULT_TABLE table=default.Table_abacee8a40094093b6e3454fedfa8ca6 [id=329d73d7b2244b42805b4b7cd0e2e45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.459599 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ac3ef5573c40b894c9b2e743ca0582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.461344 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 8f35f6dc9a33438db135d9978caf3b14 (DEFAULT_TABLE table=default.Table_f924e07646a94fd7a82006cef63a8abe [id=2d3c076f6ef54647beba4d1564fb9924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.462364 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f35f6dc9a33438db135d9978caf3b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.463593 32308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 60) took 1098 ms. Trace:
I20260809 14:30:00.463940 32308 rpcz_store.cc:276] 0809 14:29:59.365722 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.366073 (+ 351us) service_pool.cc:227] Handling call
0809 14:29:59.369530 (+ 3457us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:59.369631 (+ 101us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:00.150000 (+780369us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.463090 (+313090us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:00.463535 (+ 445us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:00.465889 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 (DEFAULT_TABLE table=default.Table_abacee8a40094093b6e3454fedfa8ca6 [id=329d73d7b2244b42805b4b7cd0e2e45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.466921 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ac3ef5573c40b894c9b2e743ca0582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.468642 32486 tablet_service.cc:1467] Processing CreateTablet for tablet b0ad3946f5de4387af3627a88b6404cf (DEFAULT_TABLE table=default.Table_bb02de675fd247e5af1ff74a6c57f2bd [id=eb07bcad144c4719ae97a233970493f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.468686 32143 tablet_service.cc:1467] Processing CreateTablet for tablet b0ad3946f5de4387af3627a88b6404cf (DEFAULT_TABLE table=default.Table_bb02de675fd247e5af1ff74a6c57f2bd [id=eb07bcad144c4719ae97a233970493f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.469174 32691 ts_tablet_manager.cc:1428] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.260s user 0.020s sys 0.045s
I20260809 14:30:00.469096 32693 raft_consensus.cc:348] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.469686 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ad3946f5de4387af3627a88b6404cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.469750 32693 raft_consensus.cc:374] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:00.469973 32693 raft_consensus.cc:729] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:00.470111 32691 tablet_bootstrap.cc:492] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:00.470497 32693 consensus_queue.cc:260] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.472375 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet a9f61933c2004a43af2c748ea445942c (table default.Table_305490483ff34e8bbb83440bf4b0ae50 [id=c1935d3bc49c405abe4d006932820224]) successfully deleted
W20260809 14:30:00.473242 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 6f36ab4503f24ca2a2a0d812c4c52468 because the tablet was not found. No further retry: Not found: Tablet not found: 6f36ab4503f24ca2a2a0d812c4c52468
W20260809 14:30:00.473806 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 810c39c60b1942e2ab1e48624aeb0ee2 because the tablet was not found. No further retry: Not found: Tablet not found: 810c39c60b1942e2ab1e48624aeb0ee2
I20260809 14:30:00.475837 32693 ts_tablet_manager.cc:1428] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.183s user 0.005s sys 0.000s
I20260809 14:30:00.477480 32307 tablet_service.cc:1467] Processing CreateTablet for tablet b0ad3946f5de4387af3627a88b6404cf (DEFAULT_TABLE table=default.Table_bb02de675fd247e5af1ff74a6c57f2bd [id=eb07bcad144c4719ae97a233970493f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.431036 32705 consensus_queue.cc:260] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:00.478400 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ad3946f5de4387af3627a88b6404cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.480736 32693 tablet_bootstrap.cc:492] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:00.481422 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ad3946f5de4387af3627a88b6404cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.485030 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 2a689385d063469595fcae82aa979bb1 (DEFAULT_TABLE table=default.Table_4488bd5aa0734b52a742a371dc2c995b [id=c19378f8e8994ef28e7b0eba079948c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.486380 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a689385d063469595fcae82aa979bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.486311 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 2a689385d063469595fcae82aa979bb1 (DEFAULT_TABLE table=default.Table_4488bd5aa0734b52a742a371dc2c995b [id=c19378f8e8994ef28e7b0eba079948c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.487272 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a689385d063469595fcae82aa979bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.498818 32138 tablet_service.cc:1467] Processing CreateTablet for tablet b6f17af7f81741c68bafa73c2c3d91af (DEFAULT_TABLE table=default.Table_ba8cab0449354ab3aa7efe707df20ab3 [id=225ab3132a00452ebedc7f1c7167ed9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.504384 32485 tablet_service.cc:1467] Processing CreateTablet for tablet b6f17af7f81741c68bafa73c2c3d91af (DEFAULT_TABLE table=default.Table_ba8cab0449354ab3aa7efe707df20ab3 [id=225ab3132a00452ebedc7f1c7167ed9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.505571 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f17af7f81741c68bafa73c2c3d91af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.508378 32691 tablet_bootstrap.cc:654] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:00.528321 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 2a689385d063469595fcae82aa979bb1 (DEFAULT_TABLE table=default.Table_4488bd5aa0734b52a742a371dc2c995b [id=c19378f8e8994ef28e7b0eba079948c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.529524 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a689385d063469595fcae82aa979bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.529775 32705 ts_tablet_manager.cc:1428] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.107s user 0.007s sys 0.000s
I20260809 14:30:00.530675 32705 tablet_bootstrap.cc:492] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:00.533420 408 leader_election.cc:288] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:00.538043 407 raft_consensus.cc:482] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:00.539273 407 raft_consensus.cc:504] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.542799 407 leader_election.cc:288] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:00.554765 32486 tablet_service.cc:1467] Processing CreateTablet for tablet ebc6ebda27b44048b5734e8798a980c2 (DEFAULT_TABLE table=default.Table_dde9e91054fa449aa8bd2d63b8783e9a [id=032899eca6f3408aaf32fa1078ac26fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.556113 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc6ebda27b44048b5734e8798a980c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.547995 32301 tablet_service.cc:1467] Processing CreateTablet for tablet b6f17af7f81741c68bafa73c2c3d91af (DEFAULT_TABLE table=default.Table_ba8cab0449354ab3aa7efe707df20ab3 [id=225ab3132a00452ebedc7f1c7167ed9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.563340 318 leader_election.cc:288] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:00.499931 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f17af7f81741c68bafa73c2c3d91af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.539630 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e846e795144e48e3b2a1eca8c1b26489" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:00.546654 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e378d7763c5046249011a6d6e0617c97" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:00.552142 32140 tablet_service.cc:1467] Processing CreateTablet for tablet ebc6ebda27b44048b5734e8798a980c2 (DEFAULT_TABLE table=default.Table_dde9e91054fa449aa8bd2d63b8783e9a [id=032899eca6f3408aaf32fa1078ac26fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.554518 32141 tablet_service.cc:1467] Processing CreateTablet for tablet d885baba6d6045e8b1ebcab6513719bb (DEFAULT_TABLE table=default.Table_6d9b31cc5d9d4976b94613e05d76120c [id=98bb45c719374b96bcb770bdf44f5553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.566609 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d885baba6d6045e8b1ebcab6513719bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.572263 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc6ebda27b44048b5734e8798a980c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.575568 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 5121e9f5a85a45d285d78f41a442769a (DEFAULT_TABLE table=default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.577003 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5121e9f5a85a45d285d78f41a442769a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.577011 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 67afcb47ffc947c5b1d98c96de333116 (DEFAULT_TABLE table=default.Table_a711da642c4642b883663fca660d7be8 [id=7e9823d249dd4c4ab3362be3b7a331b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.579419 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67afcb47ffc947c5b1d98c96de333116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.556486 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 5121e9f5a85a45d285d78f41a442769a (DEFAULT_TABLE table=default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.593544 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5121e9f5a85a45d285d78f41a442769a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:00.600682 32439 leader_election.cc:341] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:00.600979 392 raft_consensus.cc:2726] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:30:00.602182 32439 leader_election.cc:341] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:00.626483 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc8d0cad33b4778af9fe9247743c38f (DEFAULT_TABLE table=default.Table_305490483ff34e8bbb83440bf4b0ae50 [id=c1935d3bc49c405abe4d006932820224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.626758 408 raft_consensus.cc:482] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:00.627672 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc8d0cad33b4778af9fe9247743c38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.627411 408 raft_consensus.cc:504] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:00.630051 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc8d0cad33b4778af9fe9247743c38f (DEFAULT_TABLE table=default.Table_305490483ff34e8bbb83440bf4b0ae50 [id=c1935d3bc49c405abe4d006932820224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.630355 408 leader_election.cc:288] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:00.631179 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc8d0cad33b4778af9fe9247743c38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.633217 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 57ee617d858c42f7b03bd287a33eebe4 (DEFAULT_TABLE table=default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.634279 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ee617d858c42f7b03bd287a33eebe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.637451 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 57ee617d858c42f7b03bd287a33eebe4 (DEFAULT_TABLE table=default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.638792 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ee617d858c42f7b03bd287a33eebe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.640564 392 raft_consensus.cc:482] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:00.641099 392 raft_consensus.cc:504] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:00.642647 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 8b53f7d398d64701b4504db5c9c440b8 (DEFAULT_TABLE table=default.Table_487d358ef97f4504ac4d4d31e491ea25 [id=2baec8ebb5d74a6cb75f9c0eac1728e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.643428 392 leader_election.cc:288] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:00.644335 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b53f7d398d64701b4504db5c9c440b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.655793 32485 tablet_service.cc:1467] Processing CreateTablet for tablet 67afcb47ffc947c5b1d98c96de333116 (DEFAULT_TABLE table=default.Table_a711da642c4642b883663fca660d7be8 [id=7e9823d249dd4c4ab3362be3b7a331b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.656417 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce085227492e49549d3304a05a66a27d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:00.657297 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67afcb47ffc947c5b1d98c96de333116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.661610 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f17af7f81741c68bafa73c2c3d91af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.662540 32763 ts_tablet_manager.cc:1918] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:00.663020 32763 log.cc:1196] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/5951ffedd0cd481f9f425f0af3b8ad01
I20260809 14:30:00.663884 32763 ts_tablet_manager.cc:1939] T 5951ffedd0cd481f9f425f0af3b8ad01 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:00.640133 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb1dd7b6b6243b4ba3eee9b1b3b1f1c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:00.683590 32331 raft_consensus.cc:2454] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:30:00.697103 32303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 64) took 1288 ms. Trace:
I20260809 14:30:00.698479 32303 rpcz_store.cc:276] 0809 14:29:59.409093 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.520603 (+111510us) service_pool.cc:227] Handling call
0809 14:30:00.035921 (+515318us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.036020 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:00.213185 (+177165us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.695812 (+482627us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:00.697010 (+ 1198us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:00.699507 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 8b53f7d398d64701b4504db5c9c440b8 (DEFAULT_TABLE table=default.Table_487d358ef97f4504ac4d4d31e491ea25 [id=2baec8ebb5d74a6cb75f9c0eac1728e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.583716 353 raft_consensus.cc:2229] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:00.701891 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b53f7d398d64701b4504db5c9c440b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:00.712535 32095 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 98888.1us
I20260809 14:30:00.732430 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 5121e9f5a85a45d285d78f41a442769a (DEFAULT_TABLE table=default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.735163 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 55) took 2537 ms. Trace:
I20260809 14:30:00.729952 32300 tablet_service.cc:1467] Processing CreateTablet for tablet d885baba6d6045e8b1ebcab6513719bb (DEFAULT_TABLE table=default.Table_6d9b31cc5d9d4976b94613e05d76120c [id=98bb45c719374b96bcb770bdf44f5553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.739058 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5121e9f5a85a45d285d78f41a442769a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.737540 32763 rpcz_store.cc:276] 0809 14:29:58.198212 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.198443 (+ 231us) service_pool.cc:227] Handling call
0809 14:29:59.993756 (+1795313us) ts_tablet_manager.cc:1145] Deleting tablet 5951ffedd0cd481f9f425f0af3b8ad01
0809 14:30:00.015877 (+ 22121us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:00.505851 (+489974us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.662217 (+156366us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.735066 (+ 72849us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1792337}
I20260809 14:30:00.751055 31946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 57c074cef80146e28a3f3795077d047a
I20260809 14:30:00.757848 31946 catalog_manager.cc:6565] ReplaceTablet: tablet 57c074cef80146e28a3f3795077d047a deleted and replaced by tablet e74db9caa7f94059b1bba4a082fe57ef
I20260809 14:30:00.763379 32298 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc8d0cad33b4778af9fe9247743c38f (DEFAULT_TABLE table=default.Table_305490483ff34e8bbb83440bf4b0ae50 [id=c1935d3bc49c405abe4d006932820224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.766989 32483 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c074cef80146e28a3f3795077d047a with delete_type TABLET_DATA_DELETED (replaced by tablet e74db9caa7f94059b1bba4a082fe57ef) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:00.768136 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 5951ffedd0cd481f9f425f0af3b8ad01 (table default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]) successfully deleted
I20260809 14:30:00.769227 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:00.771224 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c074cef80146e28a3f3795077d047a with delete_type TABLET_DATA_DELETED (replaced by tablet e74db9caa7f94059b1bba4a082fe57ef) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:00.771728 32763 tablet_replica.cc:330] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
W20260809 14:30:00.772810 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2f171260cb764ac284aef85e4219767e has no consensus state
I20260809 14:30:00.772734 32763 raft_consensus.cc:2229] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:00.773238 32763 pending_rounds.cc:62] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:30:00.773494 32763 pending_rounds.cc:68] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626623838031872 op_type: NO_OP noop_request { }
I20260809 14:30:00.773922 32763 raft_consensus.cc:2866] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:00.774320 32763 raft_consensus.cc:2258] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:00.776280 32763 ts_tablet_manager.cc:1905] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:00.778861 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet ce085227492e49549d3304a05a66a27d
I20260809 14:30:00.781739 31937 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 122) took 1145 ms. Trace:
W20260809 14:30:00.743461 32263 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.670s user 0.007s sys 0.008s
I20260809 14:30:00.781229 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc8d0cad33b4778af9fe9247743c38f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:00.787254 31938 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 28) took 2568 ms (client timeout 1000 ms). Trace:
I20260809 14:30:00.759095 32302 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 65) took 1337 ms. Trace:
W20260809 14:30:00.790081 31938 rpcz_store.cc:269] 0809 14:29:58.218421 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.249051 (+ 30630us) service_pool.cc:227] Handling call
0809 14:30:00.787179 (+2538128us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:29:59.645580 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:29:59.646064 (+ 484us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:29:59.646091 (+ 27us) write_op.cc:435] Acquired schema lock
0809 14:29:59.646116 (+ 25us) tablet.cc:660] Decoding operations
0809 14:29:59.646456 (+ 340us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:29:59.646505 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0809 14:29:59.646540 (+ 35us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:29:59.646713 (+ 173us) tablet.cc:699] Row locks acquired
0809 14:29:59.646730 (+ 17us) write_op.cc:260] PREPARE: finished
0809 14:29:59.646837 (+ 107us) write_op.cc:270] Start()
0809 14:29:59.646927 (+ 90us) write_op.cc:276] Timestamp: P: 1786285799646792 usec, L: 0
0809 14:29:59.646945 (+ 18us) op_driver.cc:331] REPLICATION: starting
0809 14:29:59.647618 (+ 673us) log.cc:842] Serialized 533 byte log entry
0809 14:30:00.066149 (+418531us) op_driver.cc:444] REPLICATION: finished
0809 14:30:00.067239 (+ 1090us) write_op.cc:301] APPLY: starting
0809 14:30:00.067302 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:00.067401 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:00.067495 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:00.067682 (+ 187us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:00.067820 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:00.067862 (+ 42us) write_op.cc:312] APPLY: finished
0809 14:30:00.068090 (+ 228us) log.cc:842] Serialized 23 byte log entry
0809 14:30:00.068275 (+ 185us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:00.068346 (+ 71us) write_op.cc:454] Released schema lock
0809 14:30:00.068375 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:00.068586 (+ 211us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:00.068614 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":927,"apply.run_cpu_time_us":1537,"apply.run_wall_time_us":1537,"num_ops":1,"prepare.queue_time_us":9039,"prepare.run_cpu_time_us":2424,"prepare.run_wall_time_us":4650,"raft.queue_time_us":4036,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":16,"replication_time_us":419130}]]}
I20260809 14:30:00.795817 32485 tablet_service.cc:1514] Processing DeleteTablet for tablet ce085227492e49549d3304a05a66a27d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:59 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:00.797322 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 with delete_type TABLET_DATA_DELETED (replaced by tablet e28a765e1854486595683c0ac00da24c) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:00.782318 31937 rpcz_store.cc:276] 0809 14:29:59.636322 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.636592 (+ 270us) service_pool.cc:227] Handling call
0809 14:29:59.639184 (+ 2592us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:29:59.640004 (+ 820us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:29:59.640190 (+ 186us) catalog_manager.cc:2761] Locking tablets
0809 14:29:59.643355 (+ 3165us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:00.074318 (+430963us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:00.074564 (+ 246us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:00.778639 (+704075us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:00.781638 (+ 2999us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:29:59.650381 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:29:59.650806 (+ 425us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:29:59.650828 (+ 22us) write_op.cc:435] Acquired schema lock
0809 14:29:59.650840 (+ 12us) tablet.cc:660] Decoding operations
0809 14:29:59.651250 (+ 410us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:29:59.651286 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0809 14:29:59.651308 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:29:59.651782 (+ 474us) tablet.cc:699] Row locks acquired
0809 14:29:59.651798 (+ 16us) write_op.cc:260] PREPARE: finished
0809 14:29:59.651926 (+ 128us) write_op.cc:270] Start()
0809 14:29:59.652026 (+ 100us) write_op.cc:276] Timestamp: P: 1786285799651868 usec, L: 0
0809 14:29:59.652047 (+ 21us) op_driver.cc:331] REPLICATION: starting
0809 14:29:59.652679 (+ 632us) log.cc:842] Serialized 762 byte log entry
0809 14:30:00.066409 (+413730us) op_driver.cc:444] REPLICATION: finished
0809 14:30:00.068482 (+ 2073us) write_op.cc:301] APPLY: starting
0809 14:30:00.068533 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:00.068632 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:00.068765 (+ 133us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:00.069014 (+ 249us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:00.069142 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:00.069182 (+ 40us) write_op.cc:312] APPLY: finished
0809 14:30:00.069345 (+ 163us) log.cc:842] Serialized 29 byte log entry
0809 14:30:00.073790 (+ 4445us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:00.073865 (+ 75us) write_op.cc:454] Released schema lock
0809 14:30:00.073897 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:00.074706 (+ 809us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:00.074730 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1893,"apply.run_cpu_time_us":1588,"apply.run_wall_time_us":6470,"num_ops":2,"prepare.queue_time_us":3349,"prepare.run_cpu_time_us":2624,"prepare.run_wall_time_us":4209,"raft.queue_time_us":225,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":12,"replication_time_us":414293}]]}
I20260809 14:30:00.799345 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 969849d30c204937a7508e8d54dae427 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:00.794128 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 57ee617d858c42f7b03bd287a33eebe4 (DEFAULT_TABLE table=default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.802240 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 049678aadd4549d1b831174727647ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet ce085227492e49549d3304a05a66a27d) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:00.803582 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 422067edd08745bebdf59d4da704e665
I20260809 14:30:00.806376 32136 tablet_service.cc:1514] Processing DeleteTablet for tablet 422067edd08745bebdf59d4da704e665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:00 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:00.807834 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e846e795144e48e3b2a1eca8c1b26489" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:00.812556 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 422067edd08745bebdf59d4da704e665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:00 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:00.813686 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet ce085227492e49549d3304a05a66a27d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:59 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:00.814488 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_ff4d6273186b4934ae0d6b690122a9fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:00.810753 32308 tablet_service.cc:1467] Processing CreateTablet for tablet ebc6ebda27b44048b5734e8798a980c2 (DEFAULT_TABLE table=default.Table_dde9e91054fa449aa8bd2d63b8783e9a [id=032899eca6f3408aaf32fa1078ac26fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:00.816162 32491 tablet_service.cc:1467] Processing CreateTablet for tablet d885baba6d6045e8b1ebcab6513719bb (DEFAULT_TABLE table=default.Table_6d9b31cc5d9d4976b94613e05d76120c [id=98bb45c719374b96bcb770bdf44f5553]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:00.801115 32263 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.196s user 0.007s sys 0.010s
I20260809 14:30:00.824934 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_34c5f94d1c64490980d200b60a5dd583"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:00.828369 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34c5f94d1c64490980d200b60a5dd583 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:00.829509 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 8b53f7d398d64701b4504db5c9c440b8 (DEFAULT_TABLE table=default.Table_487d358ef97f4504ac4d4d31e491ea25 [id=2baec8ebb5d74a6cb75f9c0eac1728e1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:00.770540 32305 consensus_meta.cc:287] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.646s user 0.011s sys 0.006s
I20260809 14:30:00.837373 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d885baba6d6045e8b1ebcab6513719bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.849270 31948 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:00.833598 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c8eb8ad2dd406085779ed6a86eb70e" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:00.850968 32513 raft_consensus.cc:2379] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:00.850303 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e0df5c5c4f9a474d9ba2d8ab9f79c26d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:00.850960 31948 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c (table default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b]). Expected version 2 got 0
W20260809 14:30:00.853747 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0df5c5c4f9a474d9ba2d8ab9f79c26d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:00.834563 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ee617d858c42f7b03bd287a33eebe4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:00.856988 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff4d6273186b4934ae0d6b690122a9fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:00.858196 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7b2e6ca78b4fffb6f8daff1d874afe" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:00.863688 32295 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c074cef80146e28a3f3795077d047a with delete_type TABLET_DATA_DELETED (replaced by tablet e74db9caa7f94059b1bba4a082fe57ef) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:00.865315 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e378d7763c5046249011a6d6e0617c97" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:00.866130 32705 tablet_bootstrap.cc:654] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:00.869690 353 pending_rounds.cc:62] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 1 pending ops.
I20260809 14:30:00.894062 31942 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:49782 (request call id 124) took 1207 ms. Trace:
I20260809 14:30:00.873474 32295 tablet_service.cc:1514] Processing DeleteTablet for tablet 422067edd08745bebdf59d4da704e665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:00 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:00.875629 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c8eb8ad2dd406085779ed6a86eb70e" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:00.879598 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b53f7d398d64701b4504db5c9c440b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:00.898849 32439 leader_election.cc:341] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:00.880796 32303 tablet_service.cc:1467] Processing CreateTablet for tablet 67afcb47ffc947c5b1d98c96de333116 (DEFAULT_TABLE table=default.Table_a711da642c4642b883663fca660d7be8 [id=7e9823d249dd4c4ab3362be3b7a331b3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:00.853884 32260 leader_election.cc:341] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:00.905876 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d885baba6d6045e8b1ebcab6513719bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.910169 31938 master_service.cc:946] ReplaceTablet: received request to replace tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:00.918663 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 969849d30c204937a7508e8d54dae427 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:00.919541 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7b2e6ca78b4fffb6f8daff1d874afe" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:00.915436 31939 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 39) took 2474 ms (client timeout 1000 ms). Trace:
I20260809 14:30:00.923591 32443 leader_election.cc:302] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:00.924330 31939 rpcz_store.cc:269] 0809 14:29:58.440668 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.449193 (+ 8525us) service_pool.cc:227] Handling call
0809 14:29:58.513640 (+ 64447us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:00.915321 (+2401681us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":76366592}
I20260809 14:30:00.925508 376 raft_consensus.cc:2781] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:00.926198 376 raft_consensus.cc:482] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:00.926782 376 raft_consensus.cc:3037] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:00.918195 31940 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 40) took 1220 ms (client timeout 1000 ms). Trace:
W20260809 14:30:00.930446 31940 rpcz_store.cc:269] 0809 14:29:59.697677 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.697809 (+ 132us) service_pool.cc:227] Handling call
0809 14:30:00.918088 (+1220279us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":375168}
I20260809 14:30:00.931447 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e2d8ea00d247f69c14e04728519f8b" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:00.932364 32095 leader_election.cc:302] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:00.935634 32309 tablet_service.cc:1514] Processing DeleteTablet for tablet ce085227492e49549d3304a05a66a27d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:59 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:00.828199 32691 tablet_bootstrap.cc:492] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
W20260809 14:30:00.942406 32443 leader_election.cc:341] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:00.943135 32443 leader_election.cc:302] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:00.937700 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 90755us
I20260809 14:30:00.945353 408 raft_consensus.cc:2726] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:00.946349 32691 ts_tablet_manager.cc:1397] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.476s user 0.031s sys 0.016s
I20260809 14:30:00.792927 32302 rpcz_store.cc:276] 0809 14:29:59.409427 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.599376 (+189949us) service_pool.cc:227] Handling call
0809 14:29:59.880757 (+281381us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:59.880852 (+ 95us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:00.425231 (+544379us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.567798 (+142567us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:00.747039 (+179241us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 14:30:00.950680 32260 leader_election.cc:341] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:00.954968 32260 leader_election.cc:302] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:00.955472 31943 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 29) took 1448 ms (client timeout 1000 ms). Trace:
I20260809 14:30:00.956436 338 raft_consensus.cc:2781] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:00.957152 338 raft_consensus.cc:482] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:00.957537 338 raft_consensus.cc:3037] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:00.960934 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc6ebda27b44048b5734e8798a980c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:00.947551 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet 049678aadd4549d1b831174727647ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet ce085227492e49549d3304a05a66a27d) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:00.895921 353 pending_rounds.cc:68] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626637421260800 op_type: NO_OP noop_request { }
I20260809 14:30:00.969913 392 raft_consensus.cc:3037] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:00.981642 395 consensus_queue.cc:1035] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260809 14:30:00.993733 32489 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:00.989543 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f61933c2004a43af2c748ea445942c with delete_type TABLET_DATA_DELETED (replaced by tablet 2dc8d0cad33b4778af9fe9247743c38f) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:00.996322 31943 rpcz_store.cc:269] 0809 14:29:59.507102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.507273 (+ 171us) service_pool.cc:227] Handling call
0809 14:29:59.529914 (+ 22641us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:00.955372 (+1425458us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14658,"reported_tablets":12,"spinlock_wait_cycles":43208576}
I20260809 14:30:00.985257 32443 leader_election.cc:302] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:00.992555 31945 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_305490483ff34e8bbb83440bf4b0ae50" } new_table_name: "default.Table_423b0a16be834083833ae85a541ff1c4" modify_external_catalogs: true
W20260809 14:30:00.999243 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 54836.3us
I20260809 14:30:01.001557 408 raft_consensus.cc:2781] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:01.028288 408 raft_consensus.cc:482] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:01.028657 408 raft_consensus.cc:3037] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:01.025218 407 consensus_queue.cc:1035] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260809 14:30:00.894690 31942 rpcz_store.cc:276] 0809 14:29:59.686596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.686762 (+ 166us) service_pool.cc:227] Handling call
0809 14:29:59.687897 (+ 1135us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:29:59.688798 (+ 901us) catalog_manager.cc:3616] Apply alter schema
0809 14:29:59.689696 (+ 898us) catalog_manager.cc:3639] Acquired catalog manager lock
0809 14:29:59.689973 (+ 277us) catalog_manager.cc:3786] Updating metadata on disk
0809 14:30:00.077885 (+387912us) catalog_manager.cc:3817] Committing alterations to in-memory state
0809 14:30:00.893956 (+816071us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:29:59.692410 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:29:59.692768 (+ 358us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:29:59.692792 (+ 24us) write_op.cc:435] Acquired schema lock
0809 14:29:59.692804 (+ 12us) tablet.cc:660] Decoding operations
0809 14:29:59.693037 (+ 233us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:29:59.693072 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0809 14:29:59.693092 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:29:59.693301 (+ 209us) tablet.cc:699] Row locks acquired
0809 14:29:59.693313 (+ 12us) write_op.cc:260] PREPARE: finished
0809 14:29:59.693420 (+ 107us) write_op.cc:270] Start()
0809 14:29:59.693502 (+ 82us) write_op.cc:276] Timestamp: P: 1786285799693370 usec, L: 0
0809 14:29:59.693519 (+ 17us) op_driver.cc:331] REPLICATION: starting
0809 14:29:59.693996 (+ 477us) log.cc:842] Serialized 405 byte log entry
0809 14:30:00.066789 (+372793us) op_driver.cc:444] REPLICATION: finished
0809 14:30:00.069916 (+ 3127us) write_op.cc:301] APPLY: starting
0809 14:30:00.076833 (+ 6917us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b60000a01c8. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b89e361 00007ffa4ba03a3b 00007ffa4ba06064 00007ffa4b97d3bb 00007ffa4b87f343 00007ffa4b8813f8 00007ffa4b97bb14 00007ffa4b967dfa 00007ffa4b96bdd1 00007ffa4b96bd89 00007ffa4b96bd19
0809 14:30:00.076884 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:00.076969 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:00.076989 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:00.077107 (+ 118us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:00.077224 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:00.077269 (+ 45us) write_op.cc:312] APPLY: finished
0809 14:30:00.077411 (+ 142us) log.cc:842] Serialized 23 byte log entry
0809 14:30:00.077607 (+ 196us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:00.077668 (+ 61us) write_op.cc:454] Released schema lock
0809 14:30:00.077696 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:00.078174 (+ 478us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:00.078198 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2980,"apply.run_cpu_time_us":3441,"apply.run_wall_time_us":8436,"num_ops":1,"prepare.queue_time_us":986,"prepare.run_cpu_time_us":1853,"prepare.run_wall_time_us":1966,"raft.queue_time_us":1030,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":15,"replication_time_us":373204,"spinlock_wait_cycles":5832064,"thread_start_us":10287,"threads_started":1}]]}
W20260809 14:30:01.031380 32443 leader_election.cc:341] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:01.033524 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_bb02de675fd247e5af1ff74a6c57f2bd" } modify_external_catalogs: true
I20260809 14:30:01.033653 32443 leader_election.cc:302] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:01.034793 32443 leader_election.cc:341] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:01.035218 407 raft_consensus.cc:2726] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:01.034143 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_bb02de675fd247e5af1ff74a6c57f2bd" } modify_external_catalogs: true
I20260809 14:30:01.037930 32691 raft_consensus.cc:348] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.038748 32691 raft_consensus.cc:374] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:01.039047 32691 raft_consensus.cc:729] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:01.027863 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f171260cb764ac284aef85e4219767e) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:01.041895 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_a6bdfc241dfa4c51a4dd388f95341565"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:01.043728 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6bdfc241dfa4c51a4dd388f95341565 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:01.047493 426 tablet_replica.cc:330] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:01.045110 32305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 62) took 1671 ms. Trace:
I20260809 14:30:01.048926 32305 rpcz_store.cc:276] 0809 14:29:59.374235 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.375604 (+ 1369us) service_pool.cc:227] Handling call
0809 14:29:59.393137 (+ 17533us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:59.393225 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:00.123698 (+730473us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.953598 (+829900us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:01.045020 (+ 91422us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:00.978667 353 raft_consensus.cc:2866] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:01.050310 353 raft_consensus.cc:2258] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:01.059588 32693 tablet_bootstrap.cc:654] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:01.058975 338 raft_consensus.cc:504] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.067519 338 leader_election.cc:288] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:01.068257 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 01cfc9e0319748c8b0e928fc64706933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:01.055917 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_42864987ccd14c05a6909a6e1527e208" } modify_external_catalogs: true
I20260809 14:30:01.076357 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_42864987ccd14c05a6909a6e1527e208" } modify_external_catalogs: true
I20260809 14:30:01.079844 32260 leader_election.cc:302] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:01.082167 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e2d8ea00d247f69c14e04728519f8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:01.054818 32303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67afcb47ffc947c5b1d98c96de333116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.084374 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet e378d7763c5046249011a6d6e0617c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:01.090566 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:01.095368 395 raft_consensus.cc:2781] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:01.095800 395 raft_consensus.cc:482] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:01.096156 395 raft_consensus.cc:3037] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:01.135818 353 ts_tablet_manager.cc:1905] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:01.144106 392 raft_consensus.cc:2726] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:01.156992 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 197506us
I20260809 14:30:01.165207 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:01.165666 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b11cd8645a734be8a33a2ad5892e72f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:01.173017 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b11cd8645a734be8a33a2ad5892e72f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:01.182062 32705 tablet_bootstrap.cc:492] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:01.182583 32705 ts_tablet_manager.cc:1397] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.652s user 0.010s sys 0.019s
I20260809 14:30:01.187057 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 658ab8e11b37419ba2176a9175bfd936 (table default.Table_be3acab0711d448d95c162af38ba7180 [id=fa8fcb1d6ccb436080febae3ea991892]). Expected version 1 got 0
I20260809 14:30:01.188009 31946 catalog_manager.cc:5495] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:01.189690 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:01.150099 338 consensus_queue.cc:1035] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.049s
I20260809 14:30:01.191099 31946 catalog_manager.cc:5495] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:01.199123 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 (DEFAULT_TABLE table=default.Table_3c5c49afb5814baf8e9f3f33f5beac33 [id=0c17902222b34f16aab8173dbd666e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.200395 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fdaef8a95e4aa0b23018d384442d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.201148 32691 consensus_queue.cc:260] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:01.205037 31945 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dc8d0cad33b4778af9fe9247743c38f consensus state has no leader
W20260809 14:30:01.217918 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 07ac3ef5573c40b894c9b2e743ca0582 (table default.Table_abacee8a40094093b6e3454fedfa8ca6 [id=329d73d7b2244b42805b4b7cd0e2e45c], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 07ac3ef5573c40b894c9b2e743ca0582 consensus state has no leader
I20260809 14:30:01.223166 31940 master_service.cc:946] ReplaceTablet: received request to replace tablet 658ab8e11b37419ba2176a9175bfd936 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:01.225713 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_a4810f35181b4d12999365aa83bf91e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:01.228732 32691 ts_tablet_manager.cc:1428] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.282s user 0.005s sys 0.002s
I20260809 14:30:01.230531 32691 tablet_bootstrap.cc:492] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:01.237305 31937 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_34c5f94d1c64490980d200b60a5dd583" } new_table_name: "default.Table_fc6f88dc00b54ceab7215561f2a37bf9" modify_external_catalogs: true
I20260809 14:30:01.240631 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_2ebbb1898d9a4e84ab1650c408023a9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:01.244793 32705 raft_consensus.cc:348] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.245790 32705 raft_consensus.cc:374] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:01.246229 32705 raft_consensus.cc:729] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
W20260809 14:30:01.247164 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ebbb1898d9a4e84ab1650c408023a9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:01.261438 338 consensus_queue.cc:1035] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 14:30:01.267831 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4810f35181b4d12999365aa83bf91e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:01.271438 32705 consensus_queue.cc:260] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.309306 32705 ts_tablet_manager.cc:1428] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.126s user 0.010s sys 0.000s
I20260809 14:30:01.318876 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 (DEFAULT_TABLE table=default.Table_3c5c49afb5814baf8e9f3f33f5beac33 [id=0c17902222b34f16aab8173dbd666e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.320079 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fdaef8a95e4aa0b23018d384442d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.325363 32705 tablet_bootstrap.cc:492] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:01.310154 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 119520us
W20260809 14:30:01.332741 32260 leader_election.cc:341] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:01.339046 338 consensus_queue.cc:1035] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:01.344539 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 43eeed4a20d6413f873585c95bb2858a (DEFAULT_TABLE table=default.Table_c1126946af2e49c38aea0adda1c6272a [id=8b074e08bcaf4547af80d456e4c0abdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.347064 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eeed4a20d6413f873585c95bb2858a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.378320 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 (DEFAULT_TABLE table=default.Table_13a67d50c6c746e99d3c0ce47b81f6ad [id=20f2c60561964f50889f993f7a9421f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.381678 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bb4dd74afe4ace94eb1519950a83c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.277526 395 raft_consensus.cc:504] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:01.456202 376 consensus_meta.cc:287] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.529s user 0.010s sys 0.003s
I20260809 14:30:01.467816 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 (DEFAULT_TABLE table=default.Table_3c5c49afb5814baf8e9f3f33f5beac33 [id=0c17902222b34f16aab8173dbd666e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.469918 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fdaef8a95e4aa0b23018d384442d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.478683 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 43eeed4a20d6413f873585c95bb2858a (DEFAULT_TABLE table=default.Table_c1126946af2e49c38aea0adda1c6272a [id=8b074e08bcaf4547af80d456e4c0abdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.480789 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eeed4a20d6413f873585c95bb2858a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.484143 376 raft_consensus.cc:504] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.499063 426 raft_consensus.cc:2229] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:01.504045 426 raft_consensus.cc:2258] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:01.513370 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:01.515959 32439 leader_election.cc:341] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:01.495893 32443 leader_election.cc:302] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:01.524940 407 raft_consensus.cc:2726] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:01.526556 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 491750us
W20260809 14:30:01.542821 32491 consensus_meta.cc:287] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.567s user 0.006s sys 0.002s
I20260809 14:30:01.553506 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 (DEFAULT_TABLE table=default.Table_8b32f662e5154cf78a133dd5249f5f0f [id=7043672a284145c7a5c9a5d80c322ef8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.555120 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94c4cbdda4ed48a8b34728d231497d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.528515 376 leader_election.cc:288] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:01.557637 32147 tablet_service.cc:1467] Processing CreateTablet for tablet a051520649c84898ae634f58f3fb42b3 (DEFAULT_TABLE table=default.Table_b1f8694507e64827909363f73d0471b1 [id=ceb18fb1af684b32b996fb84734407c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.558985 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a051520649c84898ae634f58f3fb42b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.566533 32693 tablet_bootstrap.cc:492] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:01.567126 32693 ts_tablet_manager.cc:1397] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 1.087s user 0.017s sys 0.007s
I20260809 14:30:01.576221 32705 tablet_bootstrap.cc:654] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:01.604702 32763 ts_tablet_manager.cc:1918] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:01.610981 32763 log.cc:1196] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/6e873f3b715c424e8693dbcba0990fad
I20260809 14:30:01.620076 32146 tablet_service.cc:1467] Processing CreateTablet for tablet d9f01d25eac34d4d998dc457be6a5d9b (DEFAULT_TABLE table=default.Table_4be51cdd7c0f4caa9b2ba3719af75258 [id=f2f1c821dd6a4e1d9ebf893bc161805c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.622431 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9f01d25eac34d4d998dc457be6a5d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.629004 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 2f171260cb764ac284aef85e4219767e (DEFAULT_TABLE table=default.Table_42864987ccd14c05a6909a6e1527e208 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.630935 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f171260cb764ac284aef85e4219767e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.634847 426 ts_tablet_manager.cc:1905] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:01.639082 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 (DEFAULT_TABLE table=default.Table_8b32f662e5154cf78a133dd5249f5f0f [id=7043672a284145c7a5c9a5d80c322ef8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.640543 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 (DEFAULT_TABLE table=default.Table_13a67d50c6c746e99d3c0ce47b81f6ad [id=20f2c60561964f50889f993f7a9421f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.643085 32490 tablet_service.cc:1467] Processing CreateTablet for tablet a051520649c84898ae634f58f3fb42b3 (DEFAULT_TABLE table=default.Table_b1f8694507e64827909363f73d0471b1 [id=ceb18fb1af684b32b996fb84734407c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.644586 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94c4cbdda4ed48a8b34728d231497d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.645740 32487 tablet_service.cc:1467] Processing CreateTablet for tablet d9f01d25eac34d4d998dc457be6a5d9b (DEFAULT_TABLE table=default.Table_4be51cdd7c0f4caa9b2ba3719af75258 [id=f2f1c821dd6a4e1d9ebf893bc161805c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.647876 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9f01d25eac34d4d998dc457be6a5d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.649708 32763 ts_tablet_manager.cc:1939] T 6e873f3b715c424e8693dbcba0990fad P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:01.654876 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 56) took 3453 ms. Trace:
W20260809 14:30:01.655103 408 consensus_meta.cc:287] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.626s user 0.007s sys 0.004s
I20260809 14:30:01.655720 408 raft_consensus.cc:504] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.672158 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb1dd7b6b6243b4ba3eee9b1b3b1f1c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:01.674441 32491 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 78) took 1071 ms. Trace:
I20260809 14:30:01.668694 32763 rpcz_store.cc:276] 0809 14:29:58.201691 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.202007 (+ 316us) service_pool.cc:227] Handling call
0809 14:30:00.771530 (+2569523us) ts_tablet_manager.cc:1145] Deleting tablet 6e873f3b715c424e8693dbcba0990fad
0809 14:30:00.771592 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.381156 (+609564us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:01.604404 (+223248us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:01.654763 (+ 50359us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2566875}
I20260809 14:30:01.684969 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 6e873f3b715c424e8693dbcba0990fad (table default.Table_733ab1fa87b647d2961d60a43900c952 [id=0d4ec30e1ef44803b533141ba0e57fec]) successfully deleted
W20260809 14:30:01.629179 32443 leader_election.cc:341] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:01.687623 32443 leader_election.cc:302] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:01.689204 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 60947.4us
I20260809 14:30:01.690948 32705 tablet_bootstrap.cc:492] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:01.691426 32705 ts_tablet_manager.cc:1397] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.366s user 0.024s sys 0.002s
I20260809 14:30:01.694830 32705 raft_consensus.cc:348] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.695564 32705 raft_consensus.cc:374] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:01.700510 407 raft_consensus.cc:482] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:01.701112 407 raft_consensus.cc:504] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:01.704813 32705 raft_consensus.cc:729] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:01.705298 32305 tablet_service.cc:1467] Processing CreateTablet for tablet 43eeed4a20d6413f873585c95bb2858a (DEFAULT_TABLE table=default.Table_c1126946af2e49c38aea0adda1c6272a [id=8b074e08bcaf4547af80d456e4c0abdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.706951 32305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eeed4a20d6413f873585c95bb2858a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.707082 32705 consensus_queue.cc:260] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.711621 392 raft_consensus.cc:482] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:01.712162 392 raft_consensus.cc:504] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.713119 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a051520649c84898ae634f58f3fb42b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.714797 392 leader_election.cc:288] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:01.694990 376 raft_consensus.cc:2726] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:01.720140 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9ae593e06b4e2a949b799380c1a3e3" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:01.723326 32485 tablet_service.cc:1467] Processing CreateTablet for tablet 2f171260cb764ac284aef85e4219767e (DEFAULT_TABLE table=default.Table_42864987ccd14c05a6909a6e1527e208 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.725039 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bb4dd74afe4ace94eb1519950a83c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.725093 32705 ts_tablet_manager.cc:1428] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.033s user 0.007s sys 0.001s
I20260809 14:30:01.674891 32491 rpcz_store.cc:276] 0809 14:30:00.557048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.815479 (+258431us) service_pool.cc:227] Handling call
0809 14:30:00.836825 (+ 21346us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.836950 (+ 125us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:00.975922 (+138972us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:01.625970 (+650048us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:01.628479 (+ 2509us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:01.736790 32705 tablet_bootstrap.cc:492] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:01.695279 32439 leader_election.cc:341] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:01.737555 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 50399.4us
I20260809 14:30:01.746747 31948 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:01.765003 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f171260cb764ac284aef85e4219767e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.767551 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 57) took 3393 ms. Trace:
I20260809 14:30:01.773151 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 2f171260cb764ac284aef85e4219767e
I20260809 14:30:01.775862 32705 tablet_bootstrap.cc:654] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:01.771747 32763 rpcz_store.cc:276] 0809 14:29:58.374580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.388582 (+ 14002us) service_pool.cc:227] Handling call
0809 14:30:01.757651 (+3369069us) ts_tablet_manager.cc:1145] Deleting tablet e378d7763c5046249011a6d6e0617c97
0809 14:30:01.757747 (+ 96us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.767448 (+ 9701us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3367980}
W20260809 14:30:01.777221 32439 leader_election.cc:341] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Not found: Tablet not found: 049678aadd4549d1b831174727647ec8
I20260809 14:30:01.779340 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f171260cb764ac284aef85e4219767e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:01.779748 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet e378d7763c5046249011a6d6e0617c97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e378d7763c5046249011a6d6e0617c97 already in progress: creating tablet
I20260809 14:30:01.782475 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 58) took 3090 ms. Trace:
I20260809 14:30:01.785135 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 239ad14cb3a642e08bace0759bc0b732 (DEFAULT_TABLE table=default.Table_f332ca6d630349c9a3d8561b63890931 [id=284b10d1269c44dea39dc24aade3459c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.786860 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 239ad14cb3a642e08bace0759bc0b732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.789529 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f171260cb764ac284aef85e4219767e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:01.704319 407 leader_election.cc:288] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:30:01.783020 32763 rpcz_store.cc:276] 0809 14:29:58.692249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:58.693097 (+ 848us) service_pool.cc:227] Handling call
0809 14:30:01.781876 (+3088779us) ts_tablet_manager.cc:1145] Deleting tablet a9f61933c2004a43af2c748ea445942c
0809 14:30:01.781957 (+ 81us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.782395 (+ 438us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3085998}
W20260809 14:30:01.795455 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet a9f61933c2004a43af2c748ea445942c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9f61933c2004a43af2c748ea445942c already in progress: creating tablet
W20260809 14:30:01.694517 32443 leader_election.cc:341] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:01.797595 32443 leader_election.cc:302] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:01.799258 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 109234us
I20260809 14:30:01.800558 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 69) took 2179 ms. Trace:
I20260809 14:30:01.811971 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 with delete_type TABLET_DATA_DELETED (replaced by tablet e28a765e1854486595683c0ac00da24c) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:01.801057 32763 rpcz_store.cc:276] 0809 14:29:59.621095 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.623843 (+ 2748us) service_pool.cc:227] Handling call
0809 14:30:01.799933 (+2176090us) ts_tablet_manager.cc:1145] Deleting tablet 810c39c60b1942e2ab1e48624aeb0ee2
0809 14:30:01.799997 (+ 64us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.800428 (+ 431us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2175431}
I20260809 14:30:01.821290 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet b0ad3946f5de4387af3627a88b6404cf
I20260809 14:30:01.821437 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 68) took 2200 ms. Trace:
I20260809 14:30:01.821903 32763 rpcz_store.cc:276] 0809 14:29:59.620637 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.622848 (+ 2211us) service_pool.cc:227] Handling call
0809 14:30:01.820594 (+2197746us) ts_tablet_manager.cc:1145] Deleting tablet 6f36ab4503f24ca2a2a0d812c4c52468
0809 14:30:01.820677 (+ 83us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.821361 (+ 684us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2150341}
I20260809 14:30:01.827256 32705 tablet_bootstrap.cc:492] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:01.827672 32705 ts_tablet_manager.cc:1397] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.091s user 0.008s sys 0.010s
I20260809 14:30:01.829826 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 70) took 2158 ms. Trace:
I20260809 14:30:01.830288 32763 rpcz_store.cc:276] 0809 14:29:59.671315 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.677889 (+ 6574us) service_pool.cc:227] Handling call
0809 14:30:01.828930 (+2151041us) ts_tablet_manager.cc:1145] Deleting tablet 71aad93fcc8b42e6b55db6b3ed6b4f91
0809 14:30:01.829009 (+ 79us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.829745 (+ 736us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2150263}
W20260809 14:30:01.568295 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 154661us
I20260809 14:30:01.837868 32693 raft_consensus.cc:348] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.838195 32705 raft_consensus.cc:348] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:01.838681 32693 raft_consensus.cc:374] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:01.838960 32705 raft_consensus.cc:374] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:01.838963 32693 raft_consensus.cc:729] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:01.839224 32705 raft_consensus.cc:729] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:01.839277 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad3946f5de4387af3627a88b6404cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:01.838527 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 810c39c60b1942e2ab1e48624aeb0ee2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 810c39c60b1942e2ab1e48624aeb0ee2 already in progress: creating tablet
I20260809 14:30:01.850840 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 84) took 1089 ms. Trace:
I20260809 14:30:01.851336 32763 rpcz_store.cc:276] 0809 14:30:00.762058 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.770844 (+ 8786us) service_pool.cc:227] Handling call
0809 14:30:01.850180 (+1079336us) ts_tablet_manager.cc:1145] Deleting tablet 57c074cef80146e28a3f3795077d047a
0809 14:30:01.850243 (+ 63us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.850755 (+ 512us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1078047}
W20260809 14:30:01.856469 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 6f36ab4503f24ca2a2a0d812c4c52468 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f36ab4503f24ca2a2a0d812c4c52468 already in progress: creating tablet
I20260809 14:30:01.860865 32705 consensus_queue.cc:260] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:01.865947 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 86) took 1060 ms. Trace:
I20260809 14:30:01.871084 32705 ts_tablet_manager.cc:1428] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.043s user 0.004s sys 0.005s
I20260809 14:30:01.869740 32763 rpcz_store.cc:276] 0809 14:30:00.805907 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.806127 (+ 220us) service_pool.cc:227] Handling call
0809 14:30:01.865273 (+1059146us) ts_tablet_manager.cc:1145] Deleting tablet 422067edd08745bebdf59d4da704e665
0809 14:30:01.865344 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.865826 (+ 482us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1058231}
I20260809 14:30:01.878021 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 85) took 1086 ms. Trace:
I20260809 14:30:01.877790 31938 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c
I20260809 14:30:01.882550 31938 catalog_manager.cc:6565] ReplaceTablet: tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c deleted and replaced by tablet 9922cad6d10945d5b013a168a3496676
I20260809 14:30:01.883281 31938 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49782 (request call id 137) took 1230 ms. Trace:
I20260809 14:30:01.883638 32763 rpcz_store.cc:276] 0809 14:30:00.792126 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.813509 (+ 21383us) service_pool.cc:227] Handling call
0809 14:30:01.877515 (+1064006us) ts_tablet_manager.cc:1145] Deleting tablet ce085227492e49549d3304a05a66a27d
0809 14:30:01.877598 (+ 83us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.877965 (+ 367us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1063247}
I20260809 14:30:01.887418 32763 tablet_replica.cc:330] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:01.884377 31938 rpcz_store.cc:276] 0809 14:30:00.652515 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.910040 (+257525us) service_pool.cc:227] Handling call
0809 14:30:01.883084 (+973044us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:01.427456 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:01.427871 (+ 415us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:01.427893 (+ 22us) write_op.cc:435] Acquired schema lock
0809 14:30:01.427904 (+ 11us) tablet.cc:660] Decoding operations
0809 14:30:01.428295 (+ 391us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:01.428327 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:01.428358 (+ 31us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:01.456787 (+ 28429us) tablet.cc:699] Row locks acquired
0809 14:30:01.456811 (+ 24us) write_op.cc:260] PREPARE: finished
0809 14:30:01.456980 (+ 169us) write_op.cc:270] Start()
0809 14:30:01.457087 (+ 107us) write_op.cc:276] Timestamp: P: 1786285801456902 usec, L: 0
0809 14:30:01.457102 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 14:30:01.458055 (+ 953us) log.cc:842] Serialized 628 byte log entry
0809 14:30:01.869884 (+411829us) op_driver.cc:444] REPLICATION: finished
0809 14:30:01.872878 (+ 2994us) write_op.cc:301] APPLY: starting
0809 14:30:01.873061 (+ 183us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:01.873296 (+ 235us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:01.873525 (+ 229us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:01.874291 (+ 766us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:01.874732 (+ 441us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:01.874864 (+ 132us) write_op.cc:312] APPLY: finished
0809 14:30:01.875234 (+ 370us) log.cc:842] Serialized 29 byte log entry
0809 14:30:01.875441 (+ 207us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:01.875672 (+ 231us) write_op.cc:454] Released schema lock
0809 14:30:01.875728 (+ 56us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:01.878741 (+ 3013us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:01.878814 (+ 73us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2701,"apply.run_cpu_time_us":3422,"apply.run_wall_time_us":6245,"mutex_wait_us":36309,"num_ops":2,"prepare.queue_time_us":228203,"prepare.run_cpu_time_us":2543,"prepare.run_wall_time_us":30858,"replication_time_us":412494}]]}
I20260809 14:30:01.888240 32763 raft_consensus.cc:2229] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:01.888798 32763 raft_consensus.cc:2258] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:30:01.891393 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 57c074cef80146e28a3f3795077d047a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 57c074cef80146e28a3f3795077d047a already in progress: creating tablet
W20260809 14:30:01.892482 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 422067edd08745bebdf59d4da704e665 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 422067edd08745bebdf59d4da704e665 already in progress: creating tablet
I20260809 14:30:01.872258 32705 tablet_bootstrap.cc:492] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:01.901286 32763 ts_tablet_manager.cc:1905] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:01.839612 32693 consensus_queue.cc:260] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.927597 32705 tablet_bootstrap.cc:654] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:01.903764 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c1126946af2e49c38aea0adda1c6272a" } new_table_name: "default.Table_d6b92d6131e247fdbeda2d0138963386" modify_external_catalogs: true
I20260809 14:30:01.909457 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_43f73cfcef6546fb985efbe413dff035"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:01.912163 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_16f3bdcb253e48bf892996d7b4fbee7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:01.938349 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16f3bdcb253e48bf892996d7b4fbee7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:01.940516 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_43f73cfcef6546fb985efbe413dff035 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:01.942314 31930 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 49917.2us
I20260809 14:30:01.808560 407 raft_consensus.cc:2726] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:01.930184 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 56302.1us
W20260809 14:30:01.821859 32297 consensus_meta.cc:287] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.566s user 0.001s sys 0.009s
W20260809 14:30:01.974067 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet ce085227492e49549d3304a05a66a27d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce085227492e49549d3304a05a66a27d already in progress: creating tablet
I20260809 14:30:01.976265 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 9922cad6d10945d5b013a168a3496676) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:01.977906 31946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 38) took 1548 ms (client timeout 1000 ms). Trace:
I20260809 14:30:01.978940 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 239ad14cb3a642e08bace0759bc0b732 (DEFAULT_TABLE table=default.Table_f332ca6d630349c9a3d8561b63890931 [id=284b10d1269c44dea39dc24aade3459c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.979696 31938 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_be3acab0711d448d95c162af38ba7180" } modify_external_catalogs: true
I20260809 14:30:01.985823 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 239ad14cb3a642e08bace0759bc0b732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:01.990113 32705 tablet_bootstrap.cc:492] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:01.990571 32705 ts_tablet_manager.cc:1397] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.119s user 0.010s sys 0.008s
I20260809 14:30:01.993372 32705 raft_consensus.cc:348] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:01.994141 32705 raft_consensus.cc:374] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:01.994429 32705 raft_consensus.cc:729] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:01.995056 32705 consensus_queue.cc:260] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.003274 376 raft_consensus.cc:482] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:02.003871 376 raft_consensus.cc:504] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.006106 32705 ts_tablet_manager.cc:1428] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20260809 14:30:02.007205 32705 tablet_bootstrap.cc:492] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:02.021241 376 leader_election.cc:288] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:02.022552 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad3946f5de4387af3627a88b6404cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:02.023914 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 9922cad6d10945d5b013a168a3496676) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:02.030128 31939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_13a67d50c6c746e99d3c0ce47b81f6ad" } modify_external_catalogs: true
W20260809 14:30:02.035398 31937 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bce7aec3225a4fd89b3bfd963b64582a has no consensus state
I20260809 14:30:02.035840 32691 tablet_bootstrap.cc:654] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:02.037027 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 658ab8e11b37419ba2176a9175bfd936
I20260809 14:30:02.037613 32705 tablet_bootstrap.cc:654] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:02.038720 31940 catalog_manager.cc:6565] ReplaceTablet: tablet 658ab8e11b37419ba2176a9175bfd936 deleted and replaced by tablet a1c3aa2d41d64bed880fda2c84f8a0b3
I20260809 14:30:02.042686 31939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_13a67d50c6c746e99d3c0ce47b81f6ad" } modify_external_catalogs: true
I20260809 14:30:02.051077 31938 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_be3acab0711d448d95c162af38ba7180" } modify_external_catalogs: true
I20260809 14:30:01.812244 408 leader_election.cc:288] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:02.052618 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 658ab8e11b37419ba2176a9175bfd936 with delete_type TABLET_DATA_DELETED (replaced by tablet a1c3aa2d41d64bed880fda2c84f8a0b3) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:02.054383 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 658ab8e11b37419ba2176a9175bfd936 with delete_type TABLET_DATA_DELETED (replaced by tablet a1c3aa2d41d64bed880fda2c84f8a0b3) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.055989 32763 ts_tablet_manager.cc:1918] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:02.056393 32763 log.cc:1196] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/969849d30c204937a7508e8d54dae427
I20260809 14:30:02.057106 32763 ts_tablet_manager.cc:1939] T 969849d30c204937a7508e8d54dae427 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:02.066746 32705 tablet_bootstrap.cc:492] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:02.067288 32705 ts_tablet_manager.cc:1397] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.060s user 0.006s sys 0.005s
I20260809 14:30:02.070248 32705 raft_consensus.cc:348] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.071041 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 87) took 1161 ms. Trace:
I20260809 14:30:02.077953 32763 rpcz_store.cc:276] 0809 14:30:00.909938 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.918310 (+ 8372us) service_pool.cc:227] Handling call
0809 14:30:01.887226 (+968916us) ts_tablet_manager.cc:1145] Deleting tablet 969849d30c204937a7508e8d54dae427
0809 14:30:01.887271 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.962444 (+ 75173us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.055745 (+ 93301us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.070964 (+ 15219us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":967798}
I20260809 14:30:02.076360 32705 raft_consensus.cc:374] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:02.084389 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 88) took 1168 ms. Trace:
I20260809 14:30:02.084560 32705 raft_consensus.cc:729] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:02.084787 32763 rpcz_store.cc:276] 0809 14:30:00.916265 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.947277 (+ 31012us) service_pool.cc:227] Handling call
0809 14:30:02.079060 (+1131783us) ts_tablet_manager.cc:1145] Deleting tablet 049678aadd4549d1b831174727647ec8
0809 14:30:02.079103 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:02.084323 (+ 5220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1110149}
I20260809 14:30:02.085774 32705 consensus_queue.cc:260] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:01.978421 31946 rpcz_store.cc:269] 0809 14:30:00.429879 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.760173 (+330294us) service_pool.cc:227] Handling call
0809 14:30:00.820643 (+ 60470us) spinlock_profiling.cc:243] Waited 9.48 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:01.977694 (+1157051us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:01.256090 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:01.256555 (+ 465us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:01.256589 (+ 34us) write_op.cc:435] Acquired schema lock
0809 14:30:01.256609 (+ 20us) tablet.cc:660] Decoding operations
0809 14:30:01.257679 (+ 1070us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:01.257728 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:01.257760 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:01.258241 (+ 481us) tablet.cc:699] Row locks acquired
0809 14:30:01.258260 (+ 19us) write_op.cc:260] PREPARE: finished
0809 14:30:01.258516 (+ 256us) write_op.cc:270] Start()
0809 14:30:01.258624 (+ 108us) write_op.cc:276] Timestamp: P: 1786285801258466 usec, L: 0
0809 14:30:01.258646 (+ 22us) op_driver.cc:331] REPLICATION: starting
0809 14:30:01.259261 (+ 615us) log.cc:842] Serialized 913 byte log entry
0809 14:30:01.688963 (+429702us) op_driver.cc:444] REPLICATION: finished
0809 14:30:01.689578 (+ 615us) write_op.cc:301] APPLY: starting
0809 14:30:01.689667 (+ 89us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:01.689812 (+ 145us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:01.689932 (+ 120us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:01.690432 (+ 500us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:01.690807 (+ 375us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:01.690896 (+ 89us) write_op.cc:312] APPLY: finished
0809 14:30:01.691395 (+ 499us) log.cc:842] Serialized 29 byte log entry
0809 14:30:01.691750 (+ 355us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:01.691990 (+ 240us) write_op.cc:454] Released schema lock
0809 14:30:01.692132 (+ 142us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:01.728514 (+ 36382us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:01.728549 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":69617,"reported_tablets":12,"spinlock_wait_cycles":21794048,"child_traces":[["op",{"apply.queue_time_us":364,"apply.run_cpu_time_us":3464,"apply.run_wall_time_us":39287,"num_ops":2,"prepare.queue_time_us":49911,"prepare.run_cpu_time_us":2687,"prepare.run_wall_time_us":40569,"raft.queue_time_us":29747,"raft.run_cpu_time_us":20,"raft.run_wall_time_us":56,"replication_time_us":430190}]]}
I20260809 14:30:02.086458 32763 tablet_replica.cc:330] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:02.100423 32763 raft_consensus.cc:2229] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:02.100951 32763 raft_consensus.cc:2258] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:02.101976 32705 ts_tablet_manager.cc:1428] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.034s user 0.003s sys 0.003s
I20260809 14:30:02.102658 32763 ts_tablet_manager.cc:1905] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:02.105695 32705 tablet_bootstrap.cc:492] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:02.121543 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:02.129408 32705 tablet_bootstrap.cc:654] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:02.129863 32439 consensus_peers.cc:487] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 969849d30c204937a7508e8d54dae427. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:02.131109 32439 leader_election.cc:341] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:01.936946 32306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 72) took 1506 ms. Trace:
W20260809 14:30:02.143136 32147 consensus_meta.cc:287] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.522s user 0.004s sys 0.000s
I20260809 14:30:02.152777 31939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 79bb4dd74afe4ace94eb1519950a83c0
W20260809 14:30:01.937853 32301 consensus_meta.cc:287] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.674s user 0.006s sys 0.011s
I20260809 14:30:01.940246 32304 tablet_service.cc:1467] Processing CreateTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 (DEFAULT_TABLE table=default.Table_13a67d50c6c746e99d3c0ce47b81f6ad [id=20f2c60561964f50889f993f7a9421f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.948760 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9ae593e06b4e2a949b799380c1a3e3" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:01.950932 32302 tablet_service.cc:1467] Processing CreateTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 (DEFAULT_TABLE table=default.Table_8b32f662e5154cf78a133dd5249f5f0f [id=7043672a284145c7a5c9a5d80c322ef8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:01.953223 32307 tablet_service.cc:1467] Processing CreateTablet for tablet a051520649c84898ae634f58f3fb42b3 (DEFAULT_TABLE table=default.Table_b1f8694507e64827909363f73d0471b1 [id=ceb18fb1af684b32b996fb84734407c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.156158 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a051520649c84898ae634f58f3fb42b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.169217 32309 tablet_service.cc:1467] Processing CreateTablet for tablet d9f01d25eac34d4d998dc457be6a5d9b (DEFAULT_TABLE table=default.Table_4be51cdd7c0f4caa9b2ba3719af75258 [id=f2f1c821dd6a4e1d9ebf893bc161805c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.170456 32333 raft_consensus.cc:2379] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
W20260809 14:30:02.172219 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43eeed4a20d6413f873585c95bb2858a consensus state has no leader
I20260809 14:30:02.185082 31938 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet a1c3aa2d41d64bed880fda2c84f8a0b3
W20260809 14:30:02.188656 32303 consensus_meta.cc:287] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.913s user 0.011s sys 0.008s
I20260809 14:30:02.192243 338 raft_consensus.cc:482] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:02.193843 338 raft_consensus.cc:504] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.217770 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 969849d30c204937a7508e8d54dae427 (table default.Table_a8d4268684564e2e988b1e2bc08f779c [id=750e5ee924cc4aa0a577180b7ec93367]) successfully deleted
W20260809 14:30:02.218721 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 049678aadd4549d1b831174727647ec8 because the tablet was not found. No further retry: Not found: Tablet not found: 049678aadd4549d1b831174727647ec8
W20260809 14:30:01.968137 32299 consensus_meta.cc:287] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.639s user 0.006s sys 0.005s
I20260809 14:30:02.247388 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9f01d25eac34d4d998dc457be6a5d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.247449 32691 tablet_bootstrap.cc:492] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:02.247923 32691 ts_tablet_manager.cc:1397] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 1.018s user 0.011s sys 0.002s
W20260809 14:30:02.248919 31933 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 43eeed4a20d6413f873585c95bb2858a consensus state has no leader
I20260809 14:30:02.168363 32763 ts_tablet_manager.cc:1918] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:02.252274 32763 log.cc:1196] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/13f6c6a2ff894ec3a795f0c3f044ec17
I20260809 14:30:02.252972 32763 ts_tablet_manager.cc:1939] T 13f6c6a2ff894ec3a795f0c3f044ec17 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:02.256044 32763 tablet_replica.cc:330] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:02.172238 32705 tablet_bootstrap.cc:492] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:02.256783 32705 ts_tablet_manager.cc:1397] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.151s user 0.010s sys 0.001s
I20260809 14:30:02.254917 31938 master_service.cc:946] ReplaceTablet: received request to replace tablet 422067edd08745bebdf59d4da704e665 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:02.234287 32169 raft_consensus.cc:3037] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:02.258927 32705 raft_consensus.cc:348] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.259544 32705 raft_consensus.cc:374] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:02.259758 32705 raft_consensus.cc:729] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:02.260239 32705 consensus_queue.cc:260] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.262060 32705 ts_tablet_manager.cc:1428] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 14:30:02.262959 32705 tablet_bootstrap.cc:492] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:02.263366 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9d422f6e103b4228825e2f4398b4e1e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:02.273355 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ffe722b702a44148abf3d980648b96e4" } new_table_name: "default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29" modify_external_catalogs: true
W20260809 14:30:02.277074 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d422f6e103b4228825e2f4398b4e1e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:02.280375 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_66f580c0c2a74fcc889e318897940e5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:02.281407 32763 raft_consensus.cc:2229] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 14:30:02.281837 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_66f580c0c2a74fcc889e318897940e5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:02.281945 32763 raft_consensus.cc:2258] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:02.283823 32763 ts_tablet_manager.cc:1905] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:02.219451 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 2f171260cb764ac284aef85e4219767e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f171260cb764ac284aef85e4219767e already in progress: creating tablet
W20260809 14:30:02.292301 31930 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 72892.9us
I20260809 14:30:02.293533 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (table default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a]) successfully deleted
W20260809 14:30:02.294165 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet b0ad3946f5de4387af3627a88b6404cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0ad3946f5de4387af3627a88b6404cf already in progress: creating tablet
W20260809 14:30:02.294855 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c already in progress: creating tablet
I20260809 14:30:02.299292 32705 tablet_bootstrap.cc:654] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:02.299934 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e58ccd00bec645e4be0b92281bbcc459"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:02.302070 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e58ccd00bec645e4be0b92281bbcc459 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:02.303640 32691 raft_consensus.cc:348] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.305311 32691 raft_consensus.cc:374] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:02.305645 32691 raft_consensus.cc:729] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:30:02.238859 32137 consensus_meta.cc:287] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.552s user 0.005s sys 0.000s
I20260809 14:30:02.306280 32691 consensus_queue.cc:260] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.311825 32691 ts_tablet_manager.cc:1428] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.063s user 0.007s sys 0.002s
I20260809 14:30:02.312891 32691 tablet_bootstrap.cc:492] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:02.327265 32691 tablet_bootstrap.cc:654] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:02.250005 32260 leader_election.cc:341] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:02.329656 32260 leader_election.cc:302] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:02.331228 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 112457us
I20260809 14:30:02.351833 32295 tablet_service.cc:1467] Processing CreateTablet for tablet 2f171260cb764ac284aef85e4219767e (DEFAULT_TABLE table=default.Table_42864987ccd14c05a6909a6e1527e208 [id=032f6264a2c842f1ba6b007331ccfcc6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:02.356246 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb799e344ae544f9b4df659462bddba2 consensus state has no leader
I20260809 14:30:02.358614 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.133075 32306 rpcz_store.cc:276] 0809 14:30:00.430243 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.436541 (+ 6298us) service_pool.cc:227] Handling call
0809 14:30:00.437816 (+ 1275us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.437905 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.148897 (+710992us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:01.384423 (+235526us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:01.936830 (+552407us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:02.363420 32693 ts_tablet_manager.cc:1428] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.796s user 0.008s sys 0.004s
W20260809 14:30:02.336472 430 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s user 0.003s sys 0.014s
I20260809 14:30:02.365363 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f7fd4627a85c4f358f737fad5077415c" } modify_external_catalogs: true
I20260809 14:30:02.365845 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f7fd4627a85c4f358f737fad5077415c" } modify_external_catalogs: true
I20260809 14:30:02.372668 450 consensus_queue.cc:1035] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 14:30:02.375552 430 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s user 0.004s sys 0.014s
I20260809 14:30:02.381243 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5b7c498d7ea14448a8adbb89112bd384"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:02.383149 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b7c498d7ea14448a8adbb89112bd384 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:02.385676 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3b68d38823a54a27a2654a116d5d0430" } new_table_name: "default.Table_d425a68ae37b4c39adf325216d4359ba" modify_external_catalogs: true
I20260809 14:30:02.390918 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_2ebbb1898d9a4e84ab1650c408023a9a" } modify_external_catalogs: true
I20260809 14:30:02.391875 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_2ebbb1898d9a4e84ab1650c408023a9a" } modify_external_catalogs: true
W20260809 14:30:02.357509 32298 consensus_meta.cc:287] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.980s user 0.014s sys 0.002s
W20260809 14:30:02.357275 32300 consensus_meta.cc:287] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 1.013s user 0.009s sys 0.010s
W20260809 14:30:02.364028 32693 ts_tablet_manager.cc:1460] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Tablet startup took 796ms
W20260809 14:30:02.427728 31930 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 76636.8us
I20260809 14:30:02.430222 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:02.443641 456 raft_consensus.cc:2726] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 14:30:02.446973 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:02.456053 32302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94c4cbdda4ed48a8b34728d231497d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.462244 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd (DEFAULT_TABLE table=default.Table_b11cd8645a734be8a33a2ad5892e72f4 [id=4a72c13a29f74ba19c69e2392c122b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.463793 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d592b1c4520467791ffd9d36e0d97fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.469012 32308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 77) took 1915 ms. Trace:
I20260809 14:30:02.471186 32308 rpcz_store.cc:276] 0809 14:30:00.558379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.567336 (+ 8957us) service_pool.cc:227] Handling call
0809 14:30:00.816401 (+249065us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.816593 (+ 192us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.283900 (+467307us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.249020 (+965120us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.468953 (+219933us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":278349}
I20260809 14:30:02.482232 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f171260cb764ac284aef85e4219767e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.483177 32705 tablet_bootstrap.cc:492] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:02.483335 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f171260cb764ac284aef85e4219767e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.483634 32705 ts_tablet_manager.cc:1397] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.221s user 0.013s sys 0.003s
I20260809 14:30:02.432513 395 leader_election.cc:288] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:02.494592 31948 master_service.cc:946] ReplaceTablet: received request to replace tablet e378d7763c5046249011a6d6e0617c97 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:02.498055 32147 tablet_service.cc:1467] Processing CreateTablet for tablet f74b4deeb32941aeaa3c4b813333c14e (DEFAULT_TABLE table=default.Table_ff4d6273186b4934ae0d6b690122a9fc [id=7934506571194f3cbe62cea356e54db1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.499259 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74b4deeb32941aeaa3c4b813333c14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.507855 32763 ts_tablet_manager.cc:1918] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:02.508265 32763 log.cc:1196] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/658ab8e11b37419ba2176a9175bfd936
I20260809 14:30:02.510907 32763 ts_tablet_manager.cc:1939] T 658ab8e11b37419ba2176a9175bfd936 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:02.513491 32489 tablet_service.cc:1467] Processing CreateTablet for tablet f74b4deeb32941aeaa3c4b813333c14e (DEFAULT_TABLE table=default.Table_ff4d6273186b4934ae0d6b690122a9fc [id=7934506571194f3cbe62cea356e54db1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.514739 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74b4deeb32941aeaa3c4b813333c14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.495249 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd (DEFAULT_TABLE table=default.Table_b11cd8645a734be8a33a2ad5892e72f4 [id=4a72c13a29f74ba19c69e2392c122b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.519093 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 88644048fa4a443fac09e951d78b3063 (DEFAULT_TABLE table=default.Table_3b68d38823a54a27a2654a116d5d0430 [id=67767d6db83f4ae6ab265b854d2ad695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.519486 32297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 82) took 1717 ms. Trace:
I20260809 14:30:02.521310 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88644048fa4a443fac09e951d78b3063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.524896 32301 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 76) took 1992 ms. Trace:
I20260809 14:30:02.525338 32301 rpcz_store.cc:276] 0809 14:30:00.532906 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.533037 (+ 131us) service_pool.cc:227] Handling call
0809 14:30:00.660606 (+127569us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104702 00007f8399f9bb09 00007f8398ff8730 00007f83990dad21 00007f83990b81e3 00007f8399069869 00007f839775bc44 00007f839775bbd2 00007f839775bb01
0809 14:30:00.660809 (+ 203us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.660995 (+ 186us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.263147 (+602152us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.265401 (+1002254us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.524833 (+259432us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":165190528}
I20260809 14:30:02.526098 32705 raft_consensus.cc:348] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.526710 32705 raft_consensus.cc:374] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:02.526962 32705 raft_consensus.cc:729] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:02.527551 32705 consensus_queue.cc:260] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.533336 31945 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
I20260809 14:30:02.538756 338 leader_election.cc:288] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:02.538965 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 88644048fa4a443fac09e951d78b3063 (DEFAULT_TABLE table=default.Table_3b68d38823a54a27a2654a116d5d0430 [id=67767d6db83f4ae6ab265b854d2ad695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.541795 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 32cb7842617a4779bf110e4ea254e011 (DEFAULT_TABLE table=default.Table_e0df5c5c4f9a474d9ba2d8ab9f79c26d [id=2d493ef1e83d47da9117ecd5312dacdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.543016 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32cb7842617a4779bf110e4ea254e011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.558377 32705 ts_tablet_manager.cc:1428] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.074s user 0.007s sys 0.000s
I20260809 14:30:02.560262 32705 tablet_bootstrap.cc:492] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:02.563493 392 raft_consensus.cc:482] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:02.564173 392 raft_consensus.cc:504] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.565587 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_dde9e91054fa449aa8bd2d63b8783e9a" } new_table_name: "default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf" modify_external_catalogs: true
W20260809 14:30:02.415414 32693 ts_tablet_manager.cc:1462] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Trace:
0809 14:29:57.297217 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:57.297389 (+ 172us) service_pool.cc:227] Handling call
0809 14:29:57.299157 (+ 1768us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:57.299234 (+ 77us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:29:57.445667 (+146433us) tablet_metadata.cc:641] Metadata flushed
0809 14:29:57.688517 (+242850us) tablet_replica.cc:205] Creating consensus instance
0809 14:29:57.688896 (+ 379us) inbound_call.cc:173] Queueing success response
0809 14:30:00.476377 (+2787481us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 14:30:01.567800 (+1091423us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 14:30:01.826890 (+259090us) tablet_replica.cc:259] Starting instrumentation
0809 14:30:01.827443 (+ 553us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":234740,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":827,"tablet-open.queue_time_us":2787546}
I20260809 14:30:02.568733 392 leader_election.cc:288] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:02.571051 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 32cb7842617a4779bf110e4ea254e011 (DEFAULT_TABLE table=default.Table_e0df5c5c4f9a474d9ba2d8ab9f79c26d [id=2d493ef1e83d47da9117ecd5312dacdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.577327 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
W20260809 14:30:02.580013 32095 leader_election.cc:341] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Illegal state: Tablet not RUNNING: STOPPED
I20260809 14:30:02.581385 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d592b1c4520467791ffd9d36e0d97fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.583670 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32cb7842617a4779bf110e4ea254e011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.588733 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88644048fa4a443fac09e951d78b3063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.612964 32303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 80) took 1873 ms. Trace:
I20260809 14:30:02.613466 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac7afd531a746cab7618ead98b4c4d9 (DEFAULT_TABLE table=default.Table_a6bdfc241dfa4c51a4dd388f95341565 [id=617231ca8be54c389a6c0d821b081eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.613646 32303 rpcz_store.cc:276] 0809 14:30:00.756499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.761242 (+ 4743us) service_pool.cc:227] Handling call
0809 14:30:00.915574 (+154332us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.915724 (+ 150us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.274861 (+359137us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.296134 (+1021273us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.612860 (+316726us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":53422}
I20260809 14:30:02.615186 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac7afd531a746cab7618ead98b4c4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.616423 32693 tablet_bootstrap.cc:492] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:02.617116 32299 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 79) took 1916 ms. Trace:
I20260809 14:30:02.617672 32299 rpcz_store.cc:276] 0809 14:30:00.700580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.710509 (+ 9929us) service_pool.cc:227] Handling call
0809 14:30:00.736706 (+ 26197us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.736867 (+ 161us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.251873 (+515006us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.319429 (+1067556us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.617015 (+297586us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6884}
I20260809 14:30:02.619801 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:02.625698 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_05c045fe502848f4805ad20c9362876f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:02.627220 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05c045fe502848f4805ad20c9362876f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:02.628443 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 with delete_type TABLET_DATA_DELETED (replaced by tablet e28a765e1854486595683c0ac00da24c) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.630667 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.631667 32763 tablet_replica.cc:330] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:02.632426 32763 raft_consensus.cc:2229] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:02.633113 32763 pending_rounds.cc:62] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:30:02.633333 32763 pending_rounds.cc:68] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626637016313856 op_type: NO_OP noop_request { }
I20260809 14:30:02.633682 32763 raft_consensus.cc:2866] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:02.634027 32763 raft_consensus.cc:2258] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:02.633790 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f171260cb764ac284aef85e4219767e) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.643621 392 raft_consensus.cc:482] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:02.644371 392 raft_consensus.cc:504] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.647455 392 leader_election.cc:288] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:02.652326 32439 consensus_peers.cc:487] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:02.653574 32095 leader_election.cc:341] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: a9f61933c2004a43af2c748ea445942c
W20260809 14:30:02.654577 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 322289us
I20260809 14:30:02.665364 32763 ts_tablet_manager.cc:1905] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:02.669044 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 3705cf965d214eb794984b9c16604326
W20260809 14:30:02.672580 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 88644048fa4a443fac09e951d78b3063 consensus state has no leader
I20260809 14:30:02.675675 376 consensus_queue.cc:1035] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:02.676893 31942 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c95615ff96314c85af104870230894b5
W20260809 14:30:02.687156 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 79bb4dd74afe4ace94eb1519950a83c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79bb4dd74afe4ace94eb1519950a83c0 already in progress: creating tablet
W20260809 14:30:02.688025 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 because the tablet was not found. No further retry: Not found: Tablet not found: 13f6c6a2ff894ec3a795f0c3f044ec17
I20260809 14:30:02.699474 392 raft_consensus.cc:482] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:02.704074 426 ts_tablet_manager.cc:1918] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:02.695971 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 94c4cbdda4ed48a8b34728d231497d02 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:02.700184 392 raft_consensus.cc:504] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:02.520048 32297 rpcz_store.cc:276] 0809 14:30:00.784814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.785032 (+ 218us) service_pool.cc:227] Handling call
0809 14:30:00.800493 (+ 15461us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.800772 (+ 279us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.255646 (+454874us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.176745 (+921099us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.502167 (+325422us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:02.725586 392 leader_election.cc:288] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:02.739269 32705 tablet_bootstrap.cc:654] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:02.745378 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ebc6ebda27b44048b5734e8798a980c2 consensus state has no leader
I20260809 14:30:02.726487 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb1dd7b6b6243b4ba3eee9b1b3b1f1c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:02.746380 32332 raft_consensus.cc:3037] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:02.756071 31930 consensus_peers.cc:487] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 -> Peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Couldn't send request to peer 9d10512ac8b3406098fad1f7da012ab3. Status: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260809 14:30:02.761415 32334 raft_consensus.cc:2379] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6bed14aff53a41b2bbf5a13ce55dec5a in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:02.766999 32763 ts_tablet_manager.cc:1918] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:02.767372 32763 log.cc:1196] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/8b9ae593e06b4e2a949b799380c1a3e3
I20260809 14:30:02.768093 32763 ts_tablet_manager.cc:1939] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:02.769200 32763 tablet_replica.cc:330] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:02.733937 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bb4dd74afe4ace94eb1519950a83c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.769910 32763 raft_consensus.cc:2229] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:02.770429 32763 raft_consensus.cc:2258] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:02.774732 32137 tablet_service.cc:1467] Processing CreateTablet for tablet e74db9caa7f94059b1bba4a082fe57ef (DEFAULT_TABLE table=default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.734124 353 ts_tablet_manager.cc:1918] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:02.776193 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74db9caa7f94059b1bba4a082fe57ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.776131 353 log.cc:1196] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/71aad93fcc8b42e6b55db6b3ed6b4f91
I20260809 14:30:02.777560 353 ts_tablet_manager.cc:1939] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:02.778714 32148 tablet_service.cc:1467] Processing CreateTablet for tablet bce7aec3225a4fd89b3bfd963b64582a (DEFAULT_TABLE table=default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.781574 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce7aec3225a4fd89b3bfd963b64582a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.776959 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 9922cad6d10945d5b013a168a3496676 (DEFAULT_TABLE table=default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.785588 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9922cad6d10945d5b013a168a3496676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.777697 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet 3705cf965d214eb794984b9c16604326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.730224 32300 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 78) took 2067 ms. Trace:
I20260809 14:30:02.787083 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:02.793663 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_867a407e542b4a26af79c08ca5d0421f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:02.795315 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_867a407e542b4a26af79c08ca5d0421f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:02.789390 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad3946f5de4387af3627a88b6404cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.802788 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_63f8f4a2fe984a97bf1126564d7a82e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:02.805029 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_63f8f4a2fe984a97bf1126564d7a82e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:02.806811 32763 ts_tablet_manager.cc:1905] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:02.810961 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cfc9e0319748c8b0e928fc64706933" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:02.811733 32168 raft_consensus.cc:3037] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:02.813699 32095 leader_election.cc:341] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 71aad93fcc8b42e6b55db6b3ed6b4f91
I20260809 14:30:02.729272 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac7afd531a746cab7618ead98b4c4d9 (DEFAULT_TABLE table=default.Table_a6bdfc241dfa4c51a4dd388f95341565 [id=617231ca8be54c389a6c0d821b081eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.734688 32691 tablet_bootstrap.cc:492] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:02.749331 32484 tablet_service.cc:1467] Processing CreateTablet for tablet e74db9caa7f94059b1bba4a082fe57ef (DEFAULT_TABLE table=default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.752053 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 9922cad6d10945d5b013a168a3496676 (DEFAULT_TABLE table=default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.837251 32691 ts_tablet_manager.cc:1397] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.525s user 0.043s sys 0.055s
I20260809 14:30:02.755153 32482 tablet_service.cc:1467] Processing CreateTablet for tablet bce7aec3225a4fd89b3bfd963b64582a (DEFAULT_TABLE table=default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.838055 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac7afd531a746cab7618ead98b4c4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.758546 32481 tablet_service.cc:1514] Processing DeleteTablet for tablet 3705cf965d214eb794984b9c16604326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:02.840173 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9922cad6d10945d5b013a168a3496676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.841581 32691 raft_consensus.cc:348] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:02.842279 32691 raft_consensus.cc:374] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:02.842551 32691 raft_consensus.cc:729] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:02.801376 450 raft_consensus.cc:482] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:02.843276 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 8b9ae593e06b4e2a949b799380c1a3e3 (table default.Table_68743f87e3364e37a579888759048d44 [id=c52a212b949c4c93b9f6732bbcaaeb33]) successfully deleted
I20260809 14:30:02.845270 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce7aec3225a4fd89b3bfd963b64582a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.789891 32298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 81) took 2031 ms. Trace:
I20260809 14:30:02.843168 32691 consensus_queue.cc:260] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:02.865548 32691 ts_tablet_manager.cc:1428] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.028s user 0.001s sys 0.005s
I20260809 14:30:02.843747 450 raft_consensus.cc:504] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.867936 450 leader_election.cc:288] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:02.869278 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74db9caa7f94059b1bba4a082fe57ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.872601 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 94c4cbdda4ed48a8b34728d231497d02
I20260809 14:30:02.879626 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b11cd8645a734be8a33a2ad5892e72f4" } new_table_name: "default.Table_2aaadcb724904194bbbb379d4a2da394" modify_external_catalogs: true
I20260809 14:30:02.880632 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_971c53b113a249d28e56fd3f27219c60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:02.883513 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f" } modify_external_catalogs: true
I20260809 14:30:02.884027 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f" } modify_external_catalogs: true
I20260809 14:30:02.858017 32296 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 83) took 2045 ms. Trace:
I20260809 14:30:02.891844 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 9922cad6d10945d5b013a168a3496676) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.878638 32303 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f6c6a2ff894ec3a795f0c3f044ec17 with delete_type TABLET_DATA_DELETED (replaced by tablet e28a765e1854486595683c0ac00da24c) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.902947 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.903321 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9ae593e06b4e2a949b799380c1a3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.904579 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 01cfc9e0319748c8b0e928fc64706933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.906311 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 658ab8e11b37419ba2176a9175bfd936 with delete_type TABLET_DATA_DELETED (replaced by tablet a1c3aa2d41d64bed880fda2c84f8a0b3) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.907648 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.788558 32300 rpcz_store.cc:276] 0809 14:30:00.699936 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.700105 (+ 169us) service_pool.cc:227] Handling call
0809 14:30:00.837729 (+137624us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104702 00007f8399f9bb09 00007f8398ff8730 00007f83990dad21 00007f83990b81e3 00007f8399069869 00007f839775bc44 00007f839775bbd2 00007f839775bb01
0809 14:30:00.837845 (+ 116us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.838013 (+ 168us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.343193 (+505180us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.543791 (+1200598us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.728746 (+184955us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2394496}
I20260809 14:30:02.911162 32146 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f61933c2004a43af2c748ea445942c with delete_type TABLET_DATA_DELETED (replaced by tablet 2dc8d0cad33b4778af9fe9247743c38f) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.912554 392 raft_consensus.cc:482] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:02.913213 392 raft_consensus.cc:504] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.919935 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "810c39c60b1942e2ab1e48624aeb0ee2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:02.925366 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f61933c2004a43af2c748ea445942c with delete_type TABLET_DATA_DELETED (replaced by tablet 2dc8d0cad33b4778af9fe9247743c38f) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:02.878769 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 239ad14cb3a642e08bace0759bc0b732 (DEFAULT_TABLE table=default.Table_f332ca6d630349c9a3d8561b63890931 [id=284b10d1269c44dea39dc24aade3459c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.928015 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 239ad14cb3a642e08bace0759bc0b732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:02.930514 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_abacee8a40094093b6e3454fedfa8ca6" } modify_external_catalogs: true
I20260809 14:30:02.856105 32298 rpcz_store.cc:276] 0809 14:30:00.757270 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.757809 (+ 539us) service_pool.cc:227] Handling call
0809 14:30:00.766123 (+ 8314us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.766287 (+ 164us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.376704 (+610417us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.521361 (+1144657us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.780545 (+259184us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104f98 00007f8391104ebb 00007f8399fc48b9 00007f8398ff85d8 00007f83990bc063 00007f83990b7e2f 00007f83990b85ed 00007f8399069869 00007f839775bc44 00007f839775bbd2
0809 14:30:02.788785 (+ 8240us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9045376}
I20260809 14:30:02.885226 32296 rpcz_store.cc:276] 0809 14:30:00.817844 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.818145 (+ 301us) service_pool.cc:227] Handling call
0809 14:30:00.832234 (+ 14089us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.832440 (+ 206us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.822849 (+990409us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.760551 (+937702us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.856396 (+ 95845us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:02.941009 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 70) took 3035 ms. Trace:
I20260809 14:30:02.941829 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_abacee8a40094093b6e3454fedfa8ca6" } modify_external_catalogs: true
I20260809 14:30:02.937638 31945 catalog_manager.cc:6565] ReplaceTablet: tablet 94c4cbdda4ed48a8b34728d231497d02 deleted and replaced by tablet b393017627934fa1b740d8ba8ac62d51
I20260809 14:30:02.942844 32146 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c with delete_type TABLET_DATA_DELETED (replaced by tablet 9922cad6d10945d5b013a168a3496676) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.941706 353 rpcz_store.cc:276] 0809 14:29:59.905332 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.906358 (+ 1026us) service_pool.cc:227] Handling call
0809 14:30:00.455045 (+548687us) ts_tablet_manager.cc:1145] Deleting tablet 71aad93fcc8b42e6b55db6b3ed6b4f91
0809 14:30:00.455094 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:00.868827 (+413733us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54000e06d4. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8398016471 00007f8398099352 00007f83984af26c 00007f83990bccfa 00007f83990ece10 00007f83990d3e0d 00007f83990d3db9 00007f83990d3d49 00007f83990d3d11 00007f83990d2f3d 0000000000580784
0809 14:30:01.718136 (+849309us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.706524 (+988388us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.940860 (+234336us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":293819264,"tablet-delete.queue_time_us":338101}
I20260809 14:30:02.824067 32168 raft_consensus.cc:2454] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:02.943938 353 tablet_replica.cc:330] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:02.951016 31945 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_66f580c0c2a74fcc889e318897940e5f" } new_table_name: "default.Table_1f43d253ff624d0181e6801171d88ce4" modify_external_catalogs: true
I20260809 14:30:02.965070 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_76064ebb420d48dfb1586d58bf81d6a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:02.958727 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c074cef80146e28a3f3795077d047a with delete_type TABLET_DATA_DELETED (replaced by tablet e74db9caa7f94059b1bba4a082fe57ef) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:02.986454 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76064ebb420d48dfb1586d58bf81d6a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:02.987284 32146 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 with delete_type TABLET_DATA_DELETED (replaced by tablet b393017627934fa1b740d8ba8ac62d51) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.988055 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 01cfc9e0319748c8b0e928fc64706933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:02.990900 392 leader_election.cc:288] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:02.995679 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_971c53b113a249d28e56fd3f27219c60 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:03.025645 32691 tablet_bootstrap.cc:492] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:03.041044 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7b2e6ca78b4fffb6f8daff1d874afe" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:03.044219 32705 tablet_bootstrap.cc:492] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:03.048779 32705 ts_tablet_manager.cc:1397] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.489s user 0.007s sys 0.021s
I20260809 14:30:02.956538 32307 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 92) took 1004 ms. Trace:
I20260809 14:30:03.067586 32307 rpcz_store.cc:276] 0809 14:30:01.952257 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.952502 (+ 245us) service_pool.cc:227] Handling call
0809 14:30:02.155758 (+203256us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:02.155827 (+ 69us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:02.419627 (+263800us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.846491 (+426864us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.956472 (+109981us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:02.973554 32298 tablet_service.cc:1467] Processing CreateTablet for tablet f74b4deeb32941aeaa3c4b813333c14e (DEFAULT_TABLE table=default.Table_ff4d6273186b4934ae0d6b690122a9fc [id=7934506571194f3cbe62cea356e54db1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.069602 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74b4deeb32941aeaa3c4b813333c14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.055936 32705 raft_consensus.cc:348] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:02.952046 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 88644048fa4a443fac09e951d78b3063 (DEFAULT_TABLE table=default.Table_3b68d38823a54a27a2654a116d5d0430 [id=67767d6db83f4ae6ab265b854d2ad695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:02.998797 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "810c39c60b1942e2ab1e48624aeb0ee2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:03.070312 32705 raft_consensus.cc:374] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:03.005206 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd (DEFAULT_TABLE table=default.Table_b11cd8645a734be8a33a2ad5892e72f4 [id=4a72c13a29f74ba19c69e2392c122b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.070616 32705 raft_consensus.cc:729] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:03.020884 32303 tablet_service.cc:1467] Processing CreateTablet for tablet 32cb7842617a4779bf110e4ea254e011 (DEFAULT_TABLE table=default.Table_e0df5c5c4f9a474d9ba2d8ab9f79c26d [id=2d493ef1e83d47da9117ecd5312dacdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.023571 32300 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac7afd531a746cab7618ead98b4c4d9 (DEFAULT_TABLE table=default.Table_a6bdfc241dfa4c51a4dd388f95341565 [id=617231ca8be54c389a6c0d821b081eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.026083 32301 tablet_service.cc:1467] Processing CreateTablet for tablet e74db9caa7f94059b1bba4a082fe57ef (DEFAULT_TABLE table=default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.078078 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 9922cad6d10945d5b013a168a3496676 (DEFAULT_TABLE table=default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:03.090893 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3d592b1c4520467791ffd9d36e0d97fd consensus state has no leader
I20260809 14:30:03.095991 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet da7b2e6ca78b4fffb6f8daff1d874afe from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:03.084316 32307 tablet_service.cc:1467] Processing CreateTablet for tablet bce7aec3225a4fd89b3bfd963b64582a (DEFAULT_TABLE table=default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:03.106854 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 75515.4us
W20260809 14:30:03.108282 32260 consensus_peers.cc:487] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 658ab8e11b37419ba2176a9175bfd936. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:03.109124 32260 leader_election.cc:302] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
W20260809 14:30:03.110121 32260 leader_election.cc:341] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:03.116164 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7b2e6ca78b4fffb6f8daff1d874afe" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:03.116781 32443 leader_election.cc:302] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:03.117092 32513 raft_consensus.cc:2454] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
W20260809 14:30:03.117820 32443 raft_consensus.cc:2670] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:03.118871 32099 leader_election.cc:302] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:30:03.119432 32443 leader_election.cc:341] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 049678aadd4549d1b831174727647ec8
I20260809 14:30:03.090692 32705 consensus_queue.cc:260] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:03.119973 32099 raft_consensus.cc:2670] T 8b9ae593e06b4e2a949b799380c1a3e3 P e1f94e0d792c46c9a0b6ab42820da6eb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:03.120234 392 raft_consensus.cc:482] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:03.120857 392 raft_consensus.cc:504] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.124359 392 leader_election.cc:288] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:03.134531 32705 ts_tablet_manager.cc:1428] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.082s user 0.007s sys 0.002s
I20260809 14:30:03.135462 32705 tablet_bootstrap.cc:492] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:03.121658 32439 leader_election.cc:341] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:03.016360 426 log.cc:1196] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/57c074cef80146e28a3f3795077d047a
I20260809 14:30:03.140556 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e2d8ea00d247f69c14e04728519f8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:03.016177 32485 consensus_meta.cc:287] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.620s user 0.007s sys 0.004s
I20260809 14:30:03.145743 32485 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 102) took 1426 ms. Trace:
I20260809 14:30:03.146180 32485 rpcz_store.cc:276] 0809 14:30:01.719390 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.721999 (+ 2609us) service_pool.cc:227] Handling call
0809 14:30:01.724212 (+ 2213us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:01.724345 (+ 133us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:02.395606 (+671261us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.144881 (+749275us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.145624 (+ 743us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:03.152956 426 ts_tablet_manager.cc:1939] T 57c074cef80146e28a3f3795077d047a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:03.156970 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb1dd7b6b6243b4ba3eee9b1b3b1f1c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:03.158177 32514 raft_consensus.cc:2379] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
I20260809 14:30:03.140190 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cfc9e0319748c8b0e928fc64706933" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:03.159181 32515 raft_consensus.cc:3037] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:03.142217 32481 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 with delete_type TABLET_DATA_DELETED (replaced by tablet b393017627934fa1b740d8ba8ac62d51) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:03.045019 32309 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 93) took 1090 ms. Trace:
I20260809 14:30:03.162335 32309 rpcz_store.cc:276] 0809 14:30:02.131852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.143682 (+ 11830us) service_pool.cc:227] Handling call
0809 14:30:02.169832 (+ 26150us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:02.247044 (+ 77212us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:02.447675 (+200631us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.818133 (+370458us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.044964 (+226831us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:03.070936 32693 tablet_bootstrap.cc:654] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:03.073386 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88644048fa4a443fac09e951d78b3063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.110216 456 raft_consensus.cc:2781] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:03.036134 32332 raft_consensus.cc:2454] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:03.118629 32306 tablet_service.cc:1514] Processing DeleteTablet for tablet 3705cf965d214eb794984b9c16604326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:03.129031 32294 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 with delete_type TABLET_DATA_DELETED (replaced by tablet b393017627934fa1b740d8ba8ac62d51) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:03.141194 353 raft_consensus.cc:2229] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:03.165161 456 raft_consensus.cc:686] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
W20260809 14:30:03.171494 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 49886.1us
I20260809 14:30:03.173812 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9922cad6d10945d5b013a168a3496676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.156088 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 84) took 2391 ms. Trace:
I20260809 14:30:03.172851 456 consensus_queue.cc:237] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.185223 426 rpcz_store.cc:276] 0809 14:30:00.764978 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.766690 (+ 1712us) service_pool.cc:227] Handling call
0809 14:30:01.021107 (+254417us) ts_tablet_manager.cc:1145] Deleting tablet 57c074cef80146e28a3f3795077d047a
0809 14:30:01.037536 (+ 16429us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:01.503450 (+465914us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54000c2454. stack: 00007f555ab48033 00007f555ab47f30 00007f555a6c9534 00000000004db2b4 00000000004db249 00007f5561a54471 00007f5561ad7352 00007f5561eed26c 00007f5562afacfa 00007f5562b2ae10 00007f5562b11e0d 00007f5562b11db9 00007f5562b11d49 00007f5562b11d11 00007f5562b10f3d 0000000000580784
0809 14:30:02.309343 (+805893us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.703748 (+394405us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.156003 (+452255us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1025719552,"tablet-delete.queue_time_us":251951,"thread_start_us":222505,"threads_started":1}
I20260809 14:30:03.189417 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74db9caa7f94059b1bba4a082fe57ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.189337 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_80c122d74d1d4ff598b1f8e97c766fcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:03.192006 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80c122d74d1d4ff598b1f8e97c766fcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:03.184366 32443 leader_election.cc:302] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:03.197757 408 raft_consensus.cc:2726] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:30:03.198770 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 79379.8us
I20260809 14:30:03.199394 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d592b1c4520467791ffd9d36e0d97fd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:03.199903 32443 leader_election.cc:341] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:03.200134 353 raft_consensus.cc:2258] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:03.200477 32443 leader_election.cc:302] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:03.208319 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce7aec3225a4fd89b3bfd963b64582a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:03.162336 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 52244.9us
I20260809 14:30:03.213272 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac7afd531a746cab7618ead98b4c4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.215286 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_52d55a3e0baf4604a7f9330e03b5b3eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:03.217278 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52d55a3e0baf4604a7f9330e03b5b3eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:03.220438 32303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32cb7842617a4779bf110e4ea254e011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.221557 32302 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 91) took 1271 ms. Trace:
I20260809 14:30:03.222077 32302 rpcz_store.cc:276] 0809 14:30:01.949899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.950137 (+ 238us) service_pool.cc:227] Handling call
0809 14:30:02.455255 (+505118us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:02.455434 (+ 179us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:02.711689 (+256255us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.153549 (+441860us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.221193 (+ 67644us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104e82 00007f8399fc48b9 00007f8398ff85d8 00007f83990bc063 00007f83990b7e2f 00007f83990b85ed 00007f8399069869 00007f839775bc44 00007f839775bbd2
0809 14:30:03.221492 (+ 299us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":25859840}
I20260809 14:30:03.224480 32305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 89) took 1830 ms. Trace:
I20260809 14:30:03.224996 32305 rpcz_store.cc:276] 0809 14:30:01.394131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.564786 (+170655us) service_pool.cc:227] Handling call
0809 14:30:01.706231 (+141445us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:01.706398 (+ 167us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:02.909971 (+1203573us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.152087 (+242116us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.224069 (+ 71982us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104e82 00007f8399fc48b9 00007f8398ff85d8 00007f83990bc063 00007f83990b7e2f 00007f83990b85ed 00007f8399069869 00007f839775bc44 00007f839775bbd2
0809 14:30:03.224393 (+ 324us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":33004288}
I20260809 14:30:03.183527 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e2d8ea00d247f69c14e04728519f8b" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:03.233132 32514 raft_consensus.cc:2454] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:03.235112 426 tablet_replica.cc:330] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:03.237529 408 raft_consensus.cc:2726] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:03.244649 32095 leader_election.cc:302] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:03.245501 392 raft_consensus.cc:2781] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:03.246007 392 raft_consensus.cc:482] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:03.246529 392 raft_consensus.cc:3037] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:03.252883 32443 leader_election.cc:341] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:03.231256 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 57c074cef80146e28a3f3795077d047a (table default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]) successfully deleted
W20260809 14:30:03.254246 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet bce7aec3225a4fd89b3bfd963b64582a (table default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bce7aec3225a4fd89b3bfd963b64582a consensus state has no leader
I20260809 14:30:03.260568 32260 leader_election.cc:302] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:03.266362 435 raft_consensus.cc:2781] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:03.266860 435 raft_consensus.cc:482] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:03.270025 392 raft_consensus.cc:504] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.261693 353 ts_tablet_manager.cc:1905] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:03.277791 32443 leader_election.cc:302] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:03.267261 435 raft_consensus.cc:3037] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:03.256526 426 raft_consensus.cc:2229] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:03.211891 32310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 88) took 1894 ms. Trace:
I20260809 14:30:03.294260 32691 tablet_bootstrap.cc:654] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:03.283576 32705 tablet_bootstrap.cc:654] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:03.321058 31946 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f332ca6d630349c9a3d8561b63890931" } new_table_name: "default.Table_e7dbc14f657745d390a15586fda27677" modify_external_catalogs: true
I20260809 14:30:03.323232 31948 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:03.340891 392 leader_election.cc:288] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:03.341650 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e2d8ea00d247f69c14e04728519f8b" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:03.342495 32514 raft_consensus.cc:3037] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:03.297643 32515 raft_consensus.cc:2454] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:03.299782 408 raft_consensus.cc:2726] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:03.300840 426 raft_consensus.cc:2258] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:03.282971 32310 rpcz_store.cc:276] 0809 14:30:01.318034 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.318152 (+ 118us) service_pool.cc:227] Handling call
0809 14:30:01.319604 (+ 1452us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:01.319689 (+ 85us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:02.729082 (+1409393us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.136306 (+407224us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.210980 (+ 74674us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 14:30:03.306702 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 53876.6us
I20260809 14:30:03.377473 426 ts_tablet_manager.cc:1905] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:03.388947 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c074cef80146e28a3f3795077d047a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:03.390087 32332 raft_consensus.cc:2379] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:03.398649 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb1dd7b6b6243b4ba3eee9b1b3b1f1c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:03.399533 32332 raft_consensus.cc:2379] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
I20260809 14:30:03.394409 32443 leader_election.cc:302] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:03.417667 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e2d8ea00d247f69c14e04728519f8b" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:03.419319 32099 leader_election.cc:302] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:30:03.420550 32099 leader_election.cc:341] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: a9f61933c2004a43af2c748ea445942c
I20260809 14:30:03.420971 32099 leader_election.cc:302] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:03.423645 392 raft_consensus.cc:3037] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:03.453195 31940 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:03.453686 481 raft_consensus.cc:2781] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:03.426657 435 raft_consensus.cc:504] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.458793 481 raft_consensus.cc:686] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:03.459973 481 consensus_queue.cc:237] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.464979 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:03.472720 32763 ts_tablet_manager.cc:1918] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:03.473196 32763 log.cc:1196] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/71aad93fcc8b42e6b55db6b3ed6b4f91
I20260809 14:30:03.474306 32763 ts_tablet_manager.cc:1939] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:03.476260 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 (table default.Table_42864987ccd14c05a6909a6e1527e208 [id=032f6264a2c842f1ba6b007331ccfcc6]) successfully deleted
I20260809 14:30:03.477015 392 raft_consensus.cc:2726] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:03.477638 392 raft_consensus.cc:2726] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:03.478670 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 3705cf965d214eb794984b9c16604326 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3705cf965d214eb794984b9c16604326 already in progress: creating tablet
W20260809 14:30:03.480738 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 8b9ae593e06b4e2a949b799380c1a3e3 because the tablet was not found. No further retry: Not found: Tablet not found: 8b9ae593e06b4e2a949b799380c1a3e3
I20260809 14:30:03.494901 491 tablet.cc:1722] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:03.514178 32514 raft_consensus.cc:2454] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:03.515400 32095 leader_election.cc:302] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
W20260809 14:30:03.518890 31945 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf222c49c934d52a2642f20e8575702 has no consensus state
I20260809 14:30:03.523593 500 raft_consensus.cc:2781] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:03.524139 500 raft_consensus.cc:686] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:30:03.531054 32705 tablet_bootstrap.cc:492] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:03.531446 32705 ts_tablet_manager.cc:1397] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.396s user 0.016s sys 0.010s
I20260809 14:30:03.524974 500 consensus_queue.cc:237] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.535867 32705 raft_consensus.cc:348] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.536665 32705 raft_consensus.cc:374] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:03.536962 32705 raft_consensus.cc:729] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:03.539691 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c3a0952d48bb4b599ee7d7983393a2b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:03.541216 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c3a0952d48bb4b599ee7d7983393a2b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:03.546587 32705 consensus_queue.cc:260] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.548429 31940 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:03.548887 32705 ts_tablet_manager.cc:1428] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.017s user 0.004s sys 0.002s
I20260809 14:30:03.549692 32705 tablet_bootstrap.cc:492] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:03.554641 32705 tablet_bootstrap.cc:654] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:03.560952 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 140420us
W20260809 14:30:03.561959 32099 leader_election.cc:341] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 71aad93fcc8b42e6b55db6b3ed6b4f91
I20260809 14:30:03.562510 32099 leader_election.cc:302] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:30:03.564352 32099 raft_consensus.cc:2670] T 71aad93fcc8b42e6b55db6b3ed6b4f91 P e1f94e0d792c46c9a0b6ab42820da6eb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:03.566066 32099 leader_election.cc:302] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:03.568476 497 raft_consensus.cc:3037] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:03.572278 32763 tablet_replica.cc:330] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:03.573717 32763 raft_consensus.cc:2229] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:03.574435 32763 raft_consensus.cc:2258] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:03.460203 435 leader_election.cc:288] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:03.600149 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7b2e6ca78b4fffb6f8daff1d874afe" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:03.600944 32514 raft_consensus.cc:3037] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:03.601263 32763 ts_tablet_manager.cc:1905] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:03.602918 32705 tablet_bootstrap.cc:492] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:03.603340 32705 ts_tablet_manager.cc:1397] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.054s user 0.014s sys 0.000s
I20260809 14:30:03.605646 32705 raft_consensus.cc:348] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.606251 32705 raft_consensus.cc:374] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:03.606484 32705 raft_consensus.cc:729] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:03.606495 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7b2e6ca78b4fffb6f8daff1d874afe" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:03.607250 32169 raft_consensus.cc:3037] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:03.607120 32705 consensus_queue.cc:260] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.609089 32705 ts_tablet_manager.cc:1428] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260809 14:30:03.612596 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e2d8ea00d247f69c14e04728519f8b" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:03.621480 32691 tablet_bootstrap.cc:492] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:03.621899 32691 ts_tablet_manager.cc:1397] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.596s user 0.011s sys 0.010s
I20260809 14:30:03.621608 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f8b7cca659db4a67940b789660f281cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:03.622391 32693 tablet_bootstrap.cc:492] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:03.622772 32693 ts_tablet_manager.cc:1397] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 1.007s user 0.006s sys 0.028s
W20260809 14:30:03.623085 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8b7cca659db4a67940b789660f281cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:03.625113 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 59302.6us
W20260809 14:30:03.627815 32099 leader_election.cc:341] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:03.624202 32691 raft_consensus.cc:348] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.628881 32691 raft_consensus.cc:374] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:03.629177 32691 raft_consensus.cc:729] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:03.631330 32691 consensus_queue.cc:260] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.639402 32691 ts_tablet_manager.cc:1428] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260809 14:30:03.641645 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_7a57afa3d1b54cefb7f0406668eeed73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:03.643572 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a57afa3d1b54cefb7f0406668eeed73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:03.644474 32304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 90) took 1923 ms. Trace:
I20260809 14:30:03.645092 32304 rpcz_store.cc:276] 0809 14:30:01.852667 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.938412 (+ 85745us) service_pool.cc:227] Handling call
0809 14:30:02.455126 (+516714us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104bce 00007f839a52e3e9 00007f8398ff8a30 00007f83990dad3c 00007f83990b81e3 00007f8399069869 00007f839775bc44 00007f839775bbd2 00007f839775bb01
0809 14:30:02.555460 (+100334us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104e82 00007f8399fc48b9 00007f8398ff85d8 00007f83990b82a0 00007f8399069869 00007f839775bc44 00007f839775bbd2 00007f839775bb01 00007f839775ba7c
0809 14:30:02.615192 (+ 59732us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:02.615362 (+ 170us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:03.146363 (+531001us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.396690 (+250327us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.644360 (+247670us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":21092992}
I20260809 14:30:03.649137 32691 tablet_bootstrap.cc:492] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:03.651714 32705 tablet_bootstrap.cc:492] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:03.611860 484 tablet.cc:1722] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:03.656301 32295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 94) took 1406 ms. Trace:
I20260809 14:30:03.656910 32295 rpcz_store.cc:276] 0809 14:30:02.299961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.347893 (+ 47932us) service_pool.cc:227] Handling call
0809 14:30:02.469568 (+121675us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:02.469689 (+ 121us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:03.055563 (+585874us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.471833 (+416270us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.655897 (+184064us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104e82 00007f8399fc48b9 00007f8398ff85d8 00007f83990bc063 00007f83990b7e2f 00007f83990b85ed 00007f8399069869 00007f839775bc44 00007f839775bbd2
0809 14:30:03.656236 (+ 339us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":73721600}
W20260809 14:30:03.659526 32443 consensus_peers.cc:487] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Couldn't send request to peer 3eb0bc287b12421d89abf97ca66e4d9b. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:03.661998 32099 leader_election.cc:341] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:03.669914 32514 raft_consensus.cc:2454] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:03.677009 32299 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 96) took 1141 ms. Trace:
I20260809 14:30:03.677389 32299 rpcz_store.cc:276] 0809 14:30:02.535623 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.707024 (+171401us) service_pool.cc:227] Handling call
0809 14:30:02.927318 (+220294us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:02.927427 (+ 109us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:03.323205 (+395778us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.589744 (+266539us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.676541 (+ 86797us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104e82 00007f8399fc48b9 00007f8398ff85d8 00007f83990bc063 00007f83990b7e2f 00007f83990b85ed 00007f8399069869 00007f839775bc44 00007f839775bbd2
0809 14:30:03.676938 (+ 397us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":91802496}
I20260809 14:30:03.684834 450 raft_consensus.cc:482] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:03.685381 450 raft_consensus.cc:504] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:03.699563 32693 raft_consensus.cc:348] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:03.700721 31946 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 239ad14cb3a642e08bace0759bc0b732 consensus state has no leader
I20260809 14:30:03.700609 426 ts_tablet_manager.cc:1918] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:03.701234 426 log.cc:1196] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/ce085227492e49549d3304a05a66a27d
I20260809 14:30:03.702215 426 ts_tablet_manager.cc:1939] T ce085227492e49549d3304a05a66a27d P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:03.705967 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 85) took 2913 ms. Trace:
I20260809 14:30:03.713112 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:03.715456 32439 leader_election.cc:341] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:03.716142 450 leader_election.cc:288] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:03.706616 426 rpcz_store.cc:276] 0809 14:30:00.793006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.795597 (+ 2591us) service_pool.cc:227] Handling call
0809 14:30:03.234866 (+2439269us) ts_tablet_manager.cc:1145] Deleting tablet ce085227492e49549d3304a05a66a27d
0809 14:30:03.234913 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:03.688056 (+453143us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.700198 (+ 12142us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.705867 (+ 5669us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":38555,"tablet-delete.queue_time_us":2438181}
I20260809 14:30:03.725013 31939 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_4488bd5aa0734b52a742a371dc2c995b" } new_table_name: "default.Table_9b2c82fab8874762a05ad054b98539d8" modify_external_catalogs: true
I20260809 14:30:03.728091 426 tablet_replica.cc:330] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:03.701853 32260 leader_election.cc:302] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:03.729468 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:03.729936 435 raft_consensus.cc:2781] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:03.730681 435 raft_consensus.cc:686] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
W20260809 14:30:03.731220 32443 leader_election.cc:341] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:03.731866 32443 leader_election.cc:302] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:03.731730 435 consensus_queue.cc:237] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.704270 32295 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f171260cb764ac284aef85e4219767e) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:03.724310 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet ce085227492e49549d3304a05a66a27d (table default.Table_7b47ee94db13435cb8e6ea1c46c528f7 [id=1240bfe259ae4a2596755c9b3d3d7474]) successfully deleted
I20260809 14:30:03.747407 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 301b53eeb3be41a2b8ff2598d1b531d7 (DEFAULT_TABLE table=default.Table_9d422f6e103b4228825e2f4398b4e1e7 [id=89f12138e83d4889b2679f8b83243f1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.749301 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 301b53eeb3be41a2b8ff2598d1b531d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.756626 32763 ts_tablet_manager.cc:1918] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:03.757050 32763 log.cc:1196] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/a9f61933c2004a43af2c748ea445942c
I20260809 14:30:03.729457 32693 raft_consensus.cc:374] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:03.757812 32693 raft_consensus.cc:729] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:03.758534 32693 consensus_queue.cc:260] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:03.761720 32763 ts_tablet_manager.cc:1939] T a9f61933c2004a43af2c748ea445942c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:03.762943 32763 tablet_replica.cc:330] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:03.733588 450 raft_consensus.cc:2726] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:03.765012 32693 ts_tablet_manager.cc:1428] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.142s user 0.005s sys 0.004s
I20260809 14:30:03.771749 32295 tablet_service.cc:1467] Processing CreateTablet for tablet 301b53eeb3be41a2b8ff2598d1b531d7 (DEFAULT_TABLE table=default.Table_9d422f6e103b4228825e2f4398b4e1e7 [id=89f12138e83d4889b2679f8b83243f1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.774869 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 273ca6176a184a719538a9455afded4d (DEFAULT_TABLE table=default.Table_43f73cfcef6546fb985efbe413dff035 [id=72bef9dd3c5e43dc9e77990689fb7732]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.776554 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 0bad186f0ccf4c15a4f0b16bfffa945c (DEFAULT_TABLE table=default.Table_a4810f35181b4d12999365aa83bf91e5 [id=c0651c40a98c4aadb0e72df1c5b12317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.773391 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 301b53eeb3be41a2b8ff2598d1b531d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.780869 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bad186f0ccf4c15a4f0b16bfffa945c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.788797 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 273ca6176a184a719538a9455afded4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.796078 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 94ff6ff27f6144d08d68a5e79c12963f (DEFAULT_TABLE table=default.Table_e58ccd00bec645e4be0b92281bbcc459 [id=b17df3051f7d4322a4cf491f54bd7019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.801599 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ff6ff27f6144d08d68a5e79c12963f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.814472 353 ts_tablet_manager.cc:1918] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:03.814978 353 log.cc:1196] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/969849d30c204937a7508e8d54dae427
I20260809 14:30:03.816015 353 ts_tablet_manager.cc:1939] T 969849d30c204937a7508e8d54dae427 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:03.818367 426 raft_consensus.cc:2229] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:03.757359 32169 raft_consensus.cc:2454] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:03.826496 32705 tablet_bootstrap.cc:654] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:03.831561 32693 tablet_bootstrap.cc:492] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:03.833312 426 raft_consensus.cc:2258] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:03.826231 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet e28a765e1854486595683c0ac00da24c (table default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a]). Expected version 2 got 0
I20260809 14:30:03.841209 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 4cdfc56edcd34696a7b806b0ee34d594 (DEFAULT_TABLE table=default.Table_16f3bdcb253e48bf892996d7b4fbee7f [id=2edc29b3b38c425a99df406bfc8cc24e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.856889 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cdfc56edcd34696a7b806b0ee34d594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.856590 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 301b53eeb3be41a2b8ff2598d1b531d7 (DEFAULT_TABLE table=default.Table_9d422f6e103b4228825e2f4398b4e1e7 [id=89f12138e83d4889b2679f8b83243f1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.860577 32482 tablet_service.cc:1467] Processing CreateTablet for tablet 273ca6176a184a719538a9455afded4d (DEFAULT_TABLE table=default.Table_43f73cfcef6546fb985efbe413dff035 [id=72bef9dd3c5e43dc9e77990689fb7732]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:03.862375 31937 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet e28a765e1854486595683c0ac00da24c (table default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet e28a765e1854486595683c0ac00da24c consensus state has no leader
I20260809 14:30:03.864473 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 0bad186f0ccf4c15a4f0b16bfffa945c (DEFAULT_TABLE table=default.Table_a4810f35181b4d12999365aa83bf91e5 [id=c0651c40a98c4aadb0e72df1c5b12317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.869253 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 94ff6ff27f6144d08d68a5e79c12963f (DEFAULT_TABLE table=default.Table_e58ccd00bec645e4be0b92281bbcc459 [id=b17df3051f7d4322a4cf491f54bd7019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.870628 497 raft_consensus.cc:2726] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:03.873487 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 273ca6176a184a719538a9455afded4d (DEFAULT_TABLE table=default.Table_43f73cfcef6546fb985efbe413dff035 [id=72bef9dd3c5e43dc9e77990689fb7732]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.874342 32691 tablet_bootstrap.cc:654] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:03.875342 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 273ca6176a184a719538a9455afded4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.877511 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 0bad186f0ccf4c15a4f0b16bfffa945c (DEFAULT_TABLE table=default.Table_a4810f35181b4d12999365aa83bf91e5 [id=c0651c40a98c4aadb0e72df1c5b12317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.879098 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bad186f0ccf4c15a4f0b16bfffa945c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.881902 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 94ff6ff27f6144d08d68a5e79c12963f (DEFAULT_TABLE table=default.Table_e58ccd00bec645e4be0b92281bbcc459 [id=b17df3051f7d4322a4cf491f54bd7019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.883586 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ff6ff27f6144d08d68a5e79c12963f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.891728 32763 raft_consensus.cc:2229] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:03.894368 32307 tablet_service.cc:1467] Processing CreateTablet for tablet faf222c49c934d52a2642f20e8575702 (DEFAULT_TABLE table=default.Table_1f43d253ff624d0181e6801171d88ce4 [id=ca4bf32b107640319ee009cf2f3e4062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.895681 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf222c49c934d52a2642f20e8575702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.899641 32693 tablet_bootstrap.cc:654] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:03.900907 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 4cdfc56edcd34696a7b806b0ee34d594 (DEFAULT_TABLE table=default.Table_16f3bdcb253e48bf892996d7b4fbee7f [id=2edc29b3b38c425a99df406bfc8cc24e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.903061 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet a9f61933c2004a43af2c748ea445942c (table default.Table_423b0a16be834083833ae85a541ff1c4 [id=c1935d3bc49c405abe4d006932820224]) successfully deleted
I20260809 14:30:03.909185 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 4cdfc56edcd34696a7b806b0ee34d594 (DEFAULT_TABLE table=default.Table_16f3bdcb253e48bf892996d7b4fbee7f [id=2edc29b3b38c425a99df406bfc8cc24e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.911186 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cdfc56edcd34696a7b806b0ee34d594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.912711 32763 raft_consensus.cc:2258] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:03.914953 32763 ts_tablet_manager.cc:1905] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:03.905660 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_487d358ef97f4504ac4d4d31e491ea25" } new_table_name: "default.Table_fd0e2b9e56354188a3ab1f20bccf533a" modify_external_catalogs: true
I20260809 14:30:03.926033 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_77a290e4278b410cbfbe30feb015ef83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:03.939651 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77a290e4278b410cbfbe30feb015ef83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:03.945459 32146 tablet_service.cc:1467] Processing CreateTablet for tablet faf222c49c934d52a2642f20e8575702 (DEFAULT_TABLE table=default.Table_1f43d253ff624d0181e6801171d88ce4 [id=ca4bf32b107640319ee009cf2f3e4062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:03.946614 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf222c49c934d52a2642f20e8575702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.971392 32481 tablet_service.cc:1467] Processing CreateTablet for tablet faf222c49c934d52a2642f20e8575702 (DEFAULT_TABLE table=default.Table_1f43d253ff624d0181e6801171d88ce4 [id=ca4bf32b107640319ee009cf2f3e4062]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:03.980525 31930 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260809 14:30:03.985940 484 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47010 (request call id 87) took 2995 ms. Trace:
I20260809 14:30:04.006052 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 69) took 4101 ms. Trace:
I20260809 14:30:04.027320 353 rpcz_store.cc:276] 0809 14:29:59.904878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.022709 (+117831us) service_pool.cc:227] Handling call
0809 14:30:02.943678 (+2920969us) ts_tablet_manager.cc:1145] Deleting tablet 969849d30c204937a7508e8d54dae427
0809 14:30:02.943730 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:03.586170 (+642440us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.814181 (+228011us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.855798 (+ 41617us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b5800001898. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8390f80f11 00007f8391104851 00007f8391104fee 00007f8399fc48f9 00007f8398ff86a1 00007f83990be3b1 00007f83990c1229 00007f83990e95be 00007f83990e9589 00007f83990e9545 00007f83990db0d6
0809 14:30:04.005966 (+150168us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":35844,"spinlock_wait_cycles":13019520,"tablet-delete.queue_time_us":2821746}
I20260809 14:30:04.029778 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 849d798165144531810a03dc2b9b5f1e from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:04.033309 32705 tablet_bootstrap.cc:492] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:04.033880 32705 ts_tablet_manager.cc:1397] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.382s user 0.014s sys 0.013s
I20260809 14:30:04.037339 32705 raft_consensus.cc:348] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.043507 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_65d17735090446738a487e765aef028a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:04.046506 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf222c49c934d52a2642f20e8575702. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:04.049540 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65d17735090446738a487e765aef028a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:04.057317 31930 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 84979us
I20260809 14:30:04.069736 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 301b53eeb3be41a2b8ff2598d1b531d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:04.073380 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 273ca6176a184a719538a9455afded4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:04.077833 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bad186f0ccf4c15a4f0b16bfffa945c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:04.078018 32691 tablet_bootstrap.cc:492] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:04.078459 32691 ts_tablet_manager.cc:1397] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.430s user 0.008s sys 0.011s
I20260809 14:30:04.079362 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ff6ff27f6144d08d68a5e79c12963f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:04.081488 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 969849d30c204937a7508e8d54dae427 (table default.Table_a8d4268684564e2e988b1e2bc08f779c [id=750e5ee924cc4aa0a577180b7ec93367]) successfully deleted
I20260809 14:30:04.084869 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cdfc56edcd34696a7b806b0ee34d594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:03.986642 484 rpcz_store.cc:276] 0809 14:30:01.052658 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.056218 (+ 3560us) service_pool.cc:227] Handling call
0809 14:30:03.964526 (+2908308us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:01.369631 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:01.370354 (+ 723us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:01.370472 (+ 118us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:01.370581 (+ 109us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:01.370821 (+ 240us) alter_schema_op.cc:126] START. Timestamp: 1786285801369779
0809 14:30:01.370925 (+ 104us) op_driver.cc:331] REPLICATION: starting
0809 14:30:01.372369 (+ 1444us) log.cc:842] Serialized 173 byte log entry
0809 14:30:03.229291 (+1856922us) op_driver.cc:444] REPLICATION: finished
0809 14:30:03.604856 (+375565us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:03.693346 (+ 88490us) log.cc:842] Serialized 5 byte log entry
0809 14:30:03.937323 (+243977us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.949608 (+ 12285us) log.cc:842] Serialized 14 byte log entry
0809 14:30:03.955434 (+ 5826us) alter_schema_op.cc:71] Released schema lock
0809 14:30:03.964309 (+ 8875us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":375157,"mutex_wait_us":2876,"prepare.queue_time_us":256890,"prepare.run_cpu_time_us":21526,"prepare.run_wall_time_us":1006653,"replication_time_us":1858217,"thread_start_us":957742,"threads_started":4,"wal-append.queue_time_us":1037804,"wal-append.run_cpu_time_us":2177,"wal-append.run_wall_time_us":1024735}]]}
I20260809 14:30:04.019490 32693 tablet_bootstrap.cc:492] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:04.094252 32693 ts_tablet_manager.cc:1397] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.273s user 0.016s sys 0.000s
I20260809 14:30:04.098155 32693 raft_consensus.cc:348] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.098941 32693 raft_consensus.cc:374] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:04.099212 32693 raft_consensus.cc:729] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:30:04.101320 31939 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2a689385d063469595fcae82aa979bb1 consensus state has no leader
I20260809 14:30:04.082801 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 68) took 4275 ms. Trace:
I20260809 14:30:04.106261 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 609b3e6049ec4b0582d78037b7416710
I20260809 14:30:04.106003 353 rpcz_store.cc:276] 0809 14:29:59.837219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:59.850028 (+ 12809us) service_pool.cc:227] Handling call
0809 14:30:04.082372 (+4232344us) ts_tablet_manager.cc:1145] Deleting tablet 6e873f3b715c424e8693dbcba0990fad
0809 14:30:04.082427 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:04.082711 (+ 284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3956642}
I20260809 14:30:04.108394 353 tablet_replica.cc:330] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:04.110303 32691 raft_consensus.cc:348] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.110972 32691 raft_consensus.cc:374] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:04.111193 32691 raft_consensus.cc:729] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:30:04.113406 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 6e873f3b715c424e8693dbcba0990fad because the tablet was not found. No further retry: Not found: Tablet not found: 6e873f3b715c424e8693dbcba0990fad
I20260809 14:30:04.084730 426 ts_tablet_manager.cc:1905] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:04.111924 32691 consensus_queue.cc:260] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.122545 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet da7b2e6ca78b4fffb6f8daff1d874afe
I20260809 14:30:04.125073 31942 catalog_manager.cc:6565] ReplaceTablet: tablet da7b2e6ca78b4fffb6f8daff1d874afe deleted and replaced by tablet 37a3a71874fd44bf865b11c05e84ca84
I20260809 14:30:04.125631 31942 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49782 (request call id 173) took 1030 ms. Trace:
I20260809 14:30:04.125563 32485 tablet_service.cc:1514] Processing DeleteTablet for tablet da7b2e6ca78b4fffb6f8daff1d874afe with delete_type TABLET_DATA_DELETED (replaced by tablet 37a3a71874fd44bf865b11c05e84ca84) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:04.128962 32693 consensus_queue.cc:260] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.131982 32693 ts_tablet_manager.cc:1428] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.037s user 0.006s sys 0.001s
I20260809 14:30:04.132637 31937 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]). Expected version 1 got 0
I20260809 14:30:04.132845 32693 tablet_bootstrap.cc:492] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:04.136381 32691 ts_tablet_manager.cc:1428] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.058s user 0.006s sys 0.002s
I20260809 14:30:04.140358 31944 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 170) took 1257 ms. Trace:
I20260809 14:30:04.140715 31944 rpcz_store.cc:276] 0809 14:30:02.882889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.883080 (+ 191us) service_pool.cc:227] Handling call
0809 14:30:02.884551 (+ 1471us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:02.886276 (+ 1725us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:02.886395 (+ 119us) catalog_manager.cc:2761] Locking tablets
0809 14:30:03.740427 (+854032us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:04.105807 (+365380us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:04.106044 (+ 237us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:04.106174 (+ 130us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:04.140256 (+ 34082us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:03.768395 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:03.768779 (+ 384us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:03.768802 (+ 23us) write_op.cc:435] Acquired schema lock
0809 14:30:03.768814 (+ 12us) tablet.cc:660] Decoding operations
0809 14:30:03.769192 (+ 378us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:03.777217 (+ 8025us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:03.777242 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:03.777607 (+ 365us) tablet.cc:699] Row locks acquired
0809 14:30:03.777623 (+ 16us) write_op.cc:260] PREPARE: finished
0809 14:30:03.777746 (+ 123us) write_op.cc:270] Start()
0809 14:30:03.777829 (+ 83us) write_op.cc:276] Timestamp: P: 1786285803777690 usec, L: 0
0809 14:30:03.777845 (+ 16us) op_driver.cc:331] REPLICATION: starting
0809 14:30:03.778479 (+ 634us) log.cc:842] Serialized 762 byte log entry
0809 14:30:04.096962 (+318483us) op_driver.cc:444] REPLICATION: finished
0809 14:30:04.103081 (+ 6119us) write_op.cc:301] APPLY: starting
0809 14:30:04.103186 (+ 105us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:04.103355 (+ 169us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:04.103471 (+ 116us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:04.103971 (+ 500us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:04.104190 (+ 219us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:04.104248 (+ 58us) write_op.cc:312] APPLY: finished
0809 14:30:04.104548 (+ 300us) log.cc:842] Serialized 29 byte log entry
0809 14:30:04.104722 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:04.104830 (+ 108us) write_op.cc:454] Released schema lock
0809 14:30:04.104886 (+ 56us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:04.105243 (+ 357us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:04.105298 (+ 55us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5911,"apply.run_cpu_time_us":2496,"apply.run_wall_time_us":2496,"mutex_wait_us":14950,"num_ops":2,"prepare.queue_time_us":26119,"prepare.run_cpu_time_us":2339,"prepare.run_wall_time_us":10292,"replication_time_us":319046}]]}
I20260809 14:30:04.147296 32300 tablet_service.cc:1514] Processing DeleteTablet for tablet 609b3e6049ec4b0582d78037b7416710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:04.148427 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet da7b2e6ca78b4fffb6f8daff1d874afe with delete_type TABLET_DATA_DELETED (replaced by tablet 37a3a71874fd44bf865b11c05e84ca84) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:04.151401 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:04.153057 31943 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 167) took 1279 ms. Trace:
I20260809 14:30:04.156247 353 raft_consensus.cc:2229] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:04.156901 353 raft_consensus.cc:2258] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:04.158823 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:04.160385 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_06a4ae5a60b44618a6f88f5ad5b71ac9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:04.161540 353 ts_tablet_manager.cc:1905] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:04.162106 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06a4ae5a60b44618a6f88f5ad5b71ac9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:04.166249 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_a737e7da4d6445b7b9e29bc4a9fb13aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:04.167635 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a737e7da4d6445b7b9e29bc4a9fb13aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:04.153362 31943 rpcz_store.cc:276] 0809 14:30:02.874035 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.929948 (+ 55913us) service_pool.cc:227] Handling call
0809 14:30:02.942638 (+ 12690us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:02.973613 (+ 30975us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:02.973802 (+ 189us) catalog_manager.cc:2761] Locking tablets
0809 14:30:03.737967 (+764165us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:04.150832 (+412865us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:04.151098 (+ 266us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:04.151285 (+ 187us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:04.153002 (+ 1717us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:03.749010 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:03.749454 (+ 444us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:03.749486 (+ 32us) write_op.cc:435] Acquired schema lock
0809 14:30:03.749499 (+ 13us) tablet.cc:660] Decoding operations
0809 14:30:03.749823 (+ 324us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:03.749856 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:03.749878 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:03.750236 (+ 358us) tablet.cc:699] Row locks acquired
0809 14:30:03.750257 (+ 21us) write_op.cc:260] PREPARE: finished
0809 14:30:03.750352 (+ 95us) write_op.cc:270] Start()
0809 14:30:03.750427 (+ 75us) write_op.cc:276] Timestamp: P: 1786285803750303 usec, L: 0
0809 14:30:03.750444 (+ 17us) op_driver.cc:331] REPLICATION: starting
0809 14:30:03.751105 (+ 661us) log.cc:842] Serialized 762 byte log entry
0809 14:30:04.096681 (+345576us) op_driver.cc:444] REPLICATION: finished
0809 14:30:04.099865 (+ 3184us) write_op.cc:301] APPLY: starting
0809 14:30:04.099930 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:04.100109 (+ 179us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:04.100276 (+ 167us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:04.149412 (+ 49136us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:04.149747 (+ 335us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:04.149837 (+ 90us) write_op.cc:312] APPLY: finished
0809 14:30:04.150239 (+ 402us) log.cc:842] Serialized 29 byte log entry
0809 14:30:04.150468 (+ 229us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:04.150531 (+ 63us) write_op.cc:454] Released schema lock
0809 14:30:04.150580 (+ 49us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3022,"apply.run_cpu_time_us":2612,"apply.run_wall_time_us":73170,"mutex_wait_us":8998,"num_ops":2,"prepare.queue_time_us":9145,"prepare.run_cpu_time_us":2299,"prepare.run_wall_time_us":2303,"replication_time_us":346147}]]}
I20260809 14:30:04.193622 32485 tablet_service.cc:1514] Processing DeleteTablet for tablet 609b3e6049ec4b0582d78037b7416710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:04.194581 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:04.196892 32691 tablet_bootstrap.cc:492] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:04.204517 435 consensus_queue.cc:1035] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:04.125973 31942 rpcz_store.cc:276] 0809 14:30:03.095486 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.095681 (+ 195us) service_pool.cc:227] Handling call
0809 14:30:04.125526 (+1029845us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:03.781894 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:03.782315 (+ 421us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:03.782335 (+ 20us) write_op.cc:435] Acquired schema lock
0809 14:30:03.782345 (+ 10us) tablet.cc:660] Decoding operations
0809 14:30:03.782652 (+ 307us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:03.782686 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:03.782707 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:03.789733 (+ 7026us) tablet.cc:699] Row locks acquired
0809 14:30:03.789747 (+ 14us) write_op.cc:260] PREPARE: finished
0809 14:30:03.789850 (+ 103us) write_op.cc:270] Start()
0809 14:30:03.789928 (+ 78us) write_op.cc:276] Timestamp: P: 1786285803789798 usec, L: 0
0809 14:30:03.789945 (+ 17us) op_driver.cc:331] REPLICATION: starting
0809 14:30:03.790472 (+ 527us) log.cc:842] Serialized 628 byte log entry
0809 14:30:04.107394 (+316922us) op_driver.cc:444] REPLICATION: finished
0809 14:30:04.107722 (+ 328us) write_op.cc:301] APPLY: starting
0809 14:30:04.107820 (+ 98us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:04.108081 (+ 261us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:04.108221 (+ 140us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:04.108770 (+ 549us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:04.109102 (+ 332us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:04.109196 (+ 94us) write_op.cc:312] APPLY: finished
0809 14:30:04.109486 (+ 290us) log.cc:842] Serialized 29 byte log entry
0809 14:30:04.109704 (+ 218us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:04.109776 (+ 72us) write_op.cc:454] Released schema lock
0809 14:30:04.109846 (+ 70us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:04.110182 (+ 336us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:04.110231 (+ 49us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":124,"apply.run_cpu_time_us":2756,"apply.run_wall_time_us":2756,"num_ops":2,"prepare.queue_time_us":2709,"prepare.run_cpu_time_us":2035,"prepare.run_wall_time_us":44796,"replication_time_us":317325}]]}
I20260809 14:30:04.248663 435 consensus_queue.cc:1035] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:04.256184 32691 tablet_bootstrap.cc:654] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:04.262316 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2a689385d063469595fcae82aa979bb1 (table default.Table_9b2c82fab8874762a05ad054b98539d8 [id=c19378f8e8994ef28e7b0eba079948c3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2a689385d063469595fcae82aa979bb1 consensus state has no leader
W20260809 14:30:04.266402 32439 consensus_peers.cc:487] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:04.302562 32296 tablet_service.cc:1514] Processing DeleteTablet for tablet 969849d30c204937a7508e8d54dae427 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:04.318527 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b53f7d398d64701b4504db5c9c440b8 consensus state has no leader
W20260809 14:30:04.347010 31981 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8a20 after lost signal to thread 31907
W20260809 14:30:04.347998 31981 debug-util.cc:398] Leaking SignalData structure 0x7b08000ca3a0 after lost signal to thread 31984
I20260809 14:30:04.359710 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c3a0952d48bb4b599ee7d7983393a2b7" } new_table_name: "default.Table_6680c0954c634f329cc9a572853054d7" modify_external_catalogs: true
I20260809 14:30:04.393042 31946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_1f43d253ff624d0181e6801171d88ce4" } modify_external_catalogs: true
I20260809 14:30:04.393949 31946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_1f43d253ff624d0181e6801171d88ce4" } modify_external_catalogs: true
I20260809 14:30:04.398904 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_16f3bdcb253e48bf892996d7b4fbee7f" } modify_external_catalogs: true
I20260809 14:30:04.399673 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_16f3bdcb253e48bf892996d7b4fbee7f" } modify_external_catalogs: true
I20260809 14:30:04.423943 32693 tablet_bootstrap.cc:654] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:04.186429 32763 ts_tablet_manager.cc:1918] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:04.199951 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet 609b3e6049ec4b0582d78037b7416710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:04.210443 32140 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:04.436537 32763 log.cc:1196] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/bfb1dd7b6b6243b4ba3eee9b1b3b1f1c
I20260809 14:30:04.206895 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet da7b2e6ca78b4fffb6f8daff1d874afe with delete_type TABLET_DATA_DELETED (replaced by tablet 37a3a71874fd44bf865b11c05e84ca84) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:04.319759 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet ce085227492e49549d3304a05a66a27d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:59 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:04.434113 497 consensus_queue.cc:1035] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:04.437211 32763 ts_tablet_manager.cc:1939] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:04.437804 497 raft_consensus.cc:482] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:04.438185 497 raft_consensus.cc:504] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.439092 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 118) took 1502 ms. Trace:
I20260809 14:30:04.439414 32763 rpcz_store.cc:276] 0809 14:30:02.937232 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.942661 (+ 5429us) service_pool.cc:227] Handling call
0809 14:30:03.762772 (+820111us) ts_tablet_manager.cc:1145] Deleting tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c
0809 14:30:03.762825 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:03.892657 (+129832us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400120cd4. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181471 00007fb6c7204352 00007fb6c761a26c 00007fb6c8227cfa 00007fb6c8257e10 00007fb6c823ee0d 00007fb6c823edb9 00007fb6c823ed49 00007fb6c823ed11 00007fb6c823df3d 0000000000580784
0809 14:30:04.073553 (+180896us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.186192 (+112639us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.439033 (+252841us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":294514944,"tablet-delete.queue_time_us":819526}
I20260809 14:30:04.441205 497 leader_election.cc:288] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:04.441454 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c (table default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b]) successfully deleted
W20260809 14:30:04.436018 32148 consensus_meta.cc:287] T 301b53eeb3be41a2b8ff2598d1b531d7 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.503s user 0.006s sys 0.004s
I20260809 14:30:04.452651 32763 tablet_replica.cc:330] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:04.458334 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce085227492e49549d3304a05a66a27d" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:04.459439 32332 raft_consensus.cc:2454] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:04.461131 32296 tablet_service.cc:1514] Processing DeleteTablet for tablet e378d7763c5046249011a6d6e0617c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:04.463521 32099 leader_election.cc:302] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:04.464674 497 raft_consensus.cc:2781] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:04.465204 32763 raft_consensus.cc:2229] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:04.465246 497 raft_consensus.cc:482] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:04.465664 32763 raft_consensus.cc:2258] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:04.465946 497 raft_consensus.cc:3037] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:04.468144 32763 ts_tablet_manager.cc:1905] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:04.471501 376 raft_consensus.cc:482] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:04.474355 376 raft_consensus.cc:504] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.480878 376 leader_election.cc:288] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:04.481117 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3705cf965d214eb794984b9c16604326" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:04.483704 32443 leader_election.cc:341] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:04.442308 32705 raft_consensus.cc:374] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:04.488781 32705 raft_consensus.cc:729] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:04.489897 32705 consensus_queue.cc:260] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.492823 32705 ts_tablet_manager.cc:1428] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.458s user 0.007s sys 0.000s
I20260809 14:30:04.503875 376 consensus_queue.cc:1035] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:04.524374 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_63f8f4a2fe984a97bf1126564d7a82e5" } new_table_name: "default.Table_f00ff0058556436f9c41b2b6532452b1" modify_external_catalogs: true
I20260809 14:30:04.530136 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_853333f626c34bc18f5a1245314d7e99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:04.533389 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_853333f626c34bc18f5a1245314d7e99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:04.562938 426 ts_tablet_manager.cc:1918] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:04.563472 426 log.cc:1196] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/13f6c6a2ff894ec3a795f0c3f044ec17
I20260809 14:30:04.566684 32693 tablet_bootstrap.cc:492] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:04.572543 32693 ts_tablet_manager.cc:1397] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.440s user 0.011s sys 0.002s
I20260809 14:30:04.548974 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3705cf965d214eb794984b9c16604326" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:04.597784 32705 ts_tablet_manager.cc:1460] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Tablet startup took 563ms
W20260809 14:30:04.598117 32705 ts_tablet_manager.cc:1462] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Trace:
0809 14:29:57.377779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:29:57.405945 (+ 28166us) service_pool.cc:227] Handling call
0809 14:29:57.476144 (+ 70199us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:29:57.476346 (+ 202us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:29:57.815867 (+339521us) tablet_metadata.cc:641] Metadata flushed
0809 14:29:57.910496 (+ 94629us) tablet_replica.cc:205] Creating consensus instance
0809 14:29:57.930001 (+ 19505us) inbound_call.cc:173] Queueing success response
0809 14:30:03.651450 (+5721449us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 14:30:04.034348 (+382898us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 14:30:04.035597 (+ 1249us) tablet_replica.cc:259] Starting instrumentation
0809 14:30:04.036105 (+ 508us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":54923,"raft.run_cpu_time_us":432,"raft.run_wall_time_us":49232,"spinlock_wait_cycles":19200,"tablet-open.queue_time_us":5721657}
I20260809 14:30:04.602319 497 raft_consensus.cc:504] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.564438 426 ts_tablet_manager.cc:1939] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:04.618371 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 86) took 3824 ms. Trace:
I20260809 14:30:04.618827 426 rpcz_store.cc:276] 0809 14:30:00.793763 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.797137 (+ 3374us) service_pool.cc:227] Handling call
0809 14:30:03.726860 (+2929723us) ts_tablet_manager.cc:1145] Deleting tablet 13f6c6a2ff894ec3a795f0c3f044ec17
0809 14:30:03.727149 (+ 289us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:04.436315 (+709166us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.562632 (+126317us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.618311 (+ 55679us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2928516}
I20260809 14:30:04.622015 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 849d798165144531810a03dc2b9b5f1e
I20260809 14:30:04.623481 31945 catalog_manager.cc:6565] ReplaceTablet: tablet 849d798165144531810a03dc2b9b5f1e deleted and replaced by tablet 5be5cc6d7cb042d1b577cb7b24a95827
I20260809 14:30:04.626066 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet 849d798165144531810a03dc2b9b5f1e with delete_type TABLET_DATA_DELETED (replaced by tablet 5be5cc6d7cb042d1b577cb7b24a95827) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:04.627439 32295 tablet_service.cc:1514] Processing DeleteTablet for tablet 849d798165144531810a03dc2b9b5f1e with delete_type TABLET_DATA_DELETED (replaced by tablet 5be5cc6d7cb042d1b577cb7b24a95827) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:04.578238 32763 ts_tablet_manager.cc:1918] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260809 14:30:04.629997 532 log.cc:924] Time spent T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Append to log took a long time: real 0.054s user 0.000s sys 0.001s
I20260809 14:30:04.630336 32763 log.cc:1196] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/57c074cef80146e28a3f3795077d047a
I20260809 14:30:04.630208 435 consensus_queue.cc:1035] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:04.631141 32763 ts_tablet_manager.cc:1939] T 57c074cef80146e28a3f3795077d047a P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:04.612758 32705 tablet_bootstrap.cc:492] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:04.639518 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3e1ba4957ad24aa7adbabbab92339aa1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:04.641352 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (table default.Table_82bc1bf6e0f04a4680201cc8e2704244 [id=20d2073f354649b3b4bc4d583660c93a]) successfully deleted
I20260809 14:30:04.641795 426 tablet_replica.cc:330] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
W20260809 14:30:04.643255 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e1ba4957ad24aa7adbabbab92339aa1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:04.643530 426 raft_consensus.cc:2229] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:04.645364 426 raft_consensus.cc:2258] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:04.658581 497 leader_election.cc:288] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:04.668745 32691 tablet_bootstrap.cc:492] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:04.669245 32691 ts_tablet_manager.cc:1397] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.473s user 0.009s sys 0.011s
I20260809 14:30:04.669639 542 tablet.cc:1722] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:04.632239 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 117) took 1696 ms. Trace:
I20260809 14:30:04.694708 541 tablet.cc:1722] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:04.580350 32693 raft_consensus.cc:348] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.700681 32693 raft_consensus.cc:374] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:04.700930 32693 raft_consensus.cc:729] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:04.672209 32691 raft_consensus.cc:348] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.706952 32691 raft_consensus.cc:374] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 14:30:04.711714 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faad36657cd24202abc0573c9e3aaa90 has no consensus state
I20260809 14:30:04.716217 31946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet faf222c49c934d52a2642f20e8575702
I20260809 14:30:04.719306 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet faf222c49c934d52a2642f20e8575702 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:04.720217 32295 tablet_service.cc:1514] Processing DeleteTablet for tablet faf222c49c934d52a2642f20e8575702 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:04.728181 435 raft_consensus.cc:482] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:04.728648 435 raft_consensus.cc:504] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.707199 32691 raft_consensus.cc:729] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:04.730716 435 leader_election.cc:288] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:04.731612 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e378d7763c5046249011a6d6e0617c97" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:04.732251 32513 raft_consensus.cc:2454] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:04.730536 32691 consensus_queue.cc:260] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.733187 32260 leader_election.cc:302] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:04.733840 450 raft_consensus.cc:482] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:04.734089 435 raft_consensus.cc:2781] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:04.734496 435 raft_consensus.cc:482] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:04.734336 450 raft_consensus.cc:504] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.734786 435 raft_consensus.cc:3037] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:04.736217 450 leader_election.cc:288] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:04.737648 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a7a1c8e38748cd9a3c4ccef2aea13c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:04.739084 32443 leader_election.cc:341] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:04.743316 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5634154c6f149d3b2d36b961aee38e0 has no consensus state
I20260809 14:30:04.747514 426 ts_tablet_manager.cc:1905] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:04.753882 32691 ts_tablet_manager.cc:1428] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.084s user 0.006s sys 0.000s
I20260809 14:30:04.757014 32691 tablet_bootstrap.cc:492] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:04.762094 32691 tablet_bootstrap.cc:654] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:04.766587 435 raft_consensus.cc:504] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.779655 435 leader_election.cc:288] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:04.701870 32693 consensus_queue.cc:260] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.783914 338 raft_consensus.cc:482] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:04.784386 338 raft_consensus.cc:504] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.789528 338 leader_election.cc:288] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:04.557207 32095 leader_election.cc:341] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: ce085227492e49549d3304a05a66a27d
W20260809 14:30:04.811277 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 253841us
I20260809 14:30:04.824903 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e378d7763c5046249011a6d6e0617c97" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:04.825461 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet 849d798165144531810a03dc2b9b5f1e with delete_type TABLET_DATA_DELETED (replaced by tablet 5be5cc6d7cb042d1b577cb7b24a95827) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:04.825910 32169 raft_consensus.cc:2454] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:04.826524 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet e378d7763c5046249011a6d6e0617c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:04.827745 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet faf222c49c934d52a2642f20e8575702 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:04.829149 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e378d7763c5046249011a6d6e0617c97" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:04.829769 32169 raft_consensus.cc:3037] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:04.837260 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce085227492e49549d3304a05a66a27d" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:04.837922 32332 raft_consensus.cc:3037] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:04.844635 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b9541d0208bb4c938aff912a0df84a22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:04.779417 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e378d7763c5046249011a6d6e0617c97" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:04.848968 32513 raft_consensus.cc:3037] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:04.790216 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e2d8ea00d247f69c14e04728519f8b" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
W20260809 14:30:04.819077 32439 leader_election.cc:341] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:04.853269 32439 leader_election.cc:302] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:04.853437 32693 ts_tablet_manager.cc:1428] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.277s user 0.008s sys 0.000s
I20260809 14:30:04.848745 32332 raft_consensus.cc:2454] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:04.854576 450 raft_consensus.cc:2726] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:04.856703 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9541d0208bb4c938aff912a0df84a22 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:04.858906 32513 raft_consensus.cc:2454] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:04.859894 32260 leader_election.cc:302] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:04.860687 338 raft_consensus.cc:2781] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:04.861168 338 raft_consensus.cc:686] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:04.862066 338 consensus_queue.cc:237] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.690104 32763 rpcz_store.cc:276] 0809 14:30:02.936232 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.958535 (+ 22303us) service_pool.cc:227] Handling call
0809 14:30:04.442179 (+1483644us) ts_tablet_manager.cc:1145] Deleting tablet 57c074cef80146e28a3f3795077d047a
0809 14:30:04.452513 (+ 10334us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:04.537163 (+ 84650us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.578022 (+ 40859us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.632174 (+ 54152us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9862,"tablet-delete.queue_time_us":1472564}
I20260809 14:30:04.880712 32099 leader_election.cc:302] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:04.881063 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e2d8ea00d247f69c14e04728519f8b" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:04.882385 497 raft_consensus.cc:2781] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:04.882913 497 raft_consensus.cc:686] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:30:04.883867 497 consensus_queue.cc:237] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.885475 32332 raft_consensus.cc:3037] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:04.886451 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b0a7288bc1a843eabeb4ac3a50e99c45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:04.888108 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0a7288bc1a843eabeb4ac3a50e99c45 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:04.890399 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a7a1c8e38748cd9a3c4ccef2aea13c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:04.893347 32705 tablet_bootstrap.cc:654] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:04.897343 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 120) took 1910 ms. Trace:
I20260809 14:30:04.897732 32763 rpcz_store.cc:276] 0809 14:30:02.986848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.987017 (+ 169us) service_pool.cc:227] Handling call
0809 14:30:04.896579 (+1909562us) ts_tablet_manager.cc:1145] Deleting tablet 94c4cbdda4ed48a8b34728d231497d02
0809 14:30:04.896701 (+ 122us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:04.897145 (+ 444us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1889197}
I20260809 14:30:04.899024 32763 tablet_replica.cc:330] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
W20260809 14:30:04.899029 32095 leader_election.cc:341] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: ce085227492e49549d3304a05a66a27d
I20260809 14:30:04.900280 32763 raft_consensus.cc:2229] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:04.901618 32763 raft_consensus.cc:2258] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:04.905828 32260 leader_election.cc:302] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:04.907078 523 consensus_queue.cc:1035] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:04.908653 523 raft_consensus.cc:2726] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:04.909255 32169 raft_consensus.cc:2454] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
W20260809 14:30:04.909960 32439 leader_election.cc:341] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:04.910326 32439 leader_election.cc:302] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:04.911302 450 raft_consensus.cc:2726] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:04.914935 32691 tablet_bootstrap.cc:492] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:04.915361 32691 ts_tablet_manager.cc:1397] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.158s user 0.004s sys 0.007s
I20260809 14:30:04.915850 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_a711da642c4642b883663fca660d7be8" } new_table_name: "default.Table_b80dbb38192f49f797ea56d783e01f17" modify_external_catalogs: true
I20260809 14:30:04.917575 32691 raft_consensus.cc:348] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.923688 31946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f3f904c0b94440ddb1f6f7f733689d8c" } modify_external_catalogs: true
I20260809 14:30:04.924338 31946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f3f904c0b94440ddb1f6f7f733689d8c" } modify_external_catalogs: true
I20260809 14:30:04.933612 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 57c074cef80146e28a3f3795077d047a (table default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]) successfully deleted
I20260809 14:30:04.941604 32691 raft_consensus.cc:374] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:04.941880 32691 raft_consensus.cc:729] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:04.942549 32691 consensus_queue.cc:260] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:04.942054 497 consensus_queue.cc:1035] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260809 14:30:04.943759 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b46cfe02c4084ffbb8825b06376fc849"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:04.923120 426 ts_tablet_manager.cc:1918] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:04.950244 426 log.cc:1196] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/969849d30c204937a7508e8d54dae427
W20260809 14:30:04.950408 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b46cfe02c4084ffbb8825b06376fc849 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:04.951010 426 ts_tablet_manager.cc:1939] T 969849d30c204937a7508e8d54dae427 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:04.946858 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3d42eec9be6d469dbdede7cd0553e019" } new_table_name: "default.Table_d1d760ad86f94839bf3b5b71dd8809aa" modify_external_catalogs: true
I20260809 14:30:04.947352 32763 ts_tablet_manager.cc:1905] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:04.971488 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 94c4cbdda4ed48a8b34728d231497d02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94c4cbdda4ed48a8b34728d231497d02 already in progress: creating tablet
I20260809 14:30:04.981662 32691 ts_tablet_manager.cc:1428] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.066s user 0.002s sys 0.003s
I20260809 14:30:04.984809 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 87) took 4190 ms. Trace:
I20260809 14:30:04.985188 426 rpcz_store.cc:276] 0809 14:30:00.794526 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.798443 (+ 3917us) service_pool.cc:227] Handling call
0809 14:30:04.641520 (+3843077us) ts_tablet_manager.cc:1145] Deleting tablet 969849d30c204937a7508e8d54dae427
0809 14:30:04.641590 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:04.823729 (+182139us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.922899 (+ 99170us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.984745 (+ 61846us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31371,"tablet-delete.queue_time_us":3841418}
I20260809 14:30:04.986766 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 969849d30c204937a7508e8d54dae427 (table default.Table_a8d4268684564e2e988b1e2bc08f779c [id=750e5ee924cc4aa0a577180b7ec93367]) successfully deleted
I20260809 14:30:04.991129 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_0227da888bb24888976eae18fce543e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:04.992653 32691 tablet_bootstrap.cc:492] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:30:04.992820 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0227da888bb24888976eae18fce543e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:05.003003 426 tablet_replica.cc:330] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:05.004770 426 raft_consensus.cc:2229] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:05.005298 426 raft_consensus.cc:2258] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.007081 426 ts_tablet_manager.cc:1905] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.021027 32705 tablet_bootstrap.cc:492] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:05.018110 32691 tablet_bootstrap.cc:654] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:05.038667 557 tablet.cc:1722] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:05.021611 32705 ts_tablet_manager.cc:1397] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.409s user 0.008s sys 0.011s
I20260809 14:30:05.047966 32693 tablet_bootstrap.cc:492] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:04.979480 353 ts_tablet_manager.cc:1918] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:05.058530 353 log.cc:1196] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/57c074cef80146e28a3f3795077d047a
I20260809 14:30:05.076673 426 ts_tablet_manager.cc:1918] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:05.081985 32705 raft_consensus.cc:348] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.082796 32705 raft_consensus.cc:374] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:05.083140 32705 raft_consensus.cc:729] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:05.083765 32705 consensus_queue.cc:260] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.085882 32705 ts_tablet_manager.cc:1428] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.041s user 0.004s sys 0.001s
I20260809 14:30:05.089259 32705 tablet_bootstrap.cc:492] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:05.096346 426 log.cc:1196] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/049678aadd4549d1b831174727647ec8
W20260809 14:30:05.088085 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 48992.8us
I20260809 14:30:05.106691 353 ts_tablet_manager.cc:1939] T 57c074cef80146e28a3f3795077d047a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:05.107529 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 84) took 4268 ms. Trace:
I20260809 14:30:05.097131 426 ts_tablet_manager.cc:1939] T 049678aadd4549d1b831174727647ec8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:05.116916 32693 tablet_bootstrap.cc:654] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:05.107854 353 rpcz_store.cc:276] 0809 14:30:00.849736 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.850162 (+ 426us) service_pool.cc:227] Handling call
0809 14:30:04.107883 (+3257721us) ts_tablet_manager.cc:1145] Deleting tablet 57c074cef80146e28a3f3795077d047a
0809 14:30:04.107977 (+ 94us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:04.601807 (+493830us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:04.979231 (+377424us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.107458 (+128227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3243351}
I20260809 14:30:05.119788 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 57c074cef80146e28a3f3795077d047a (table default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]) successfully deleted
I20260809 14:30:05.120954 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 86) took 4248 ms. Trace:
I20260809 14:30:05.102712 32763 ts_tablet_manager.cc:1918] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:05.122426 32763 log.cc:1196] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/01cfc9e0319748c8b0e928fc64706933
I20260809 14:30:05.123546 32763 ts_tablet_manager.cc:1939] T 01cfc9e0319748c8b0e928fc64706933 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:05.126364 353 rpcz_store.cc:276] 0809 14:30:00.872924 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.873067 (+ 143us) service_pool.cc:227] Handling call
0809 14:30:05.120590 (+4247523us) ts_tablet_manager.cc:1145] Deleting tablet 422067edd08745bebdf59d4da704e665
0809 14:30:05.120635 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.120895 (+ 260us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4224198}
I20260809 14:30:05.116998 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 88) took 4322 ms. Trace:
I20260809 14:30:05.130828 426 rpcz_store.cc:276] 0809 14:30:00.795357 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.801444 (+ 6087us) service_pool.cc:227] Handling call
0809 14:30:05.002485 (+4201041us) ts_tablet_manager.cc:1145] Deleting tablet 049678aadd4549d1b831174727647ec8
0809 14:30:05.002536 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.050653 (+ 48117us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.064269 (+ 13616us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.116922 (+ 52653us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4199356}
W20260809 14:30:05.133088 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67afcb47ffc947c5b1d98c96de333116 consensus state has no leader
I20260809 14:30:05.133778 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet 27e2d8ea00d247f69c14e04728519f8b from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:05.135445 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet 84c8eb8ad2dd406085779ed6a86eb70e from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:05.138139 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 89) took 4325 ms. Trace:
I20260809 14:30:05.133323 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 049678aadd4549d1b831174727647ec8 (table default.Table_7b47ee94db13435cb8e6ea1c46c528f7 [id=1240bfe259ae4a2596755c9b3d3d7474]) successfully deleted
I20260809 14:30:05.147296 32705 tablet_bootstrap.cc:654] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:05.151574 32691 tablet_bootstrap.cc:492] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:05.151973 32691 ts_tablet_manager.cc:1397] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.160s user 0.015s sys 0.005s
I20260809 14:30:05.140295 426 rpcz_store.cc:276] 0809 14:30:00.812153 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.812322 (+ 169us) service_pool.cc:227] Handling call
0809 14:30:05.135229 (+4322907us) ts_tablet_manager.cc:1145] Deleting tablet 422067edd08745bebdf59d4da704e665
0809 14:30:05.135658 (+ 429us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.137832 (+ 2174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4321816}
W20260809 14:30:05.153396 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 422067edd08745bebdf59d4da704e665 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 422067edd08745bebdf59d4da704e665 already in progress: creating tablet
I20260809 14:30:05.154431 426 tablet_replica.cc:330] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
W20260809 14:30:05.128337 32260 consensus_peers.cc:487] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01cfc9e0319748c8b0e928fc64706933. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:05.156100 426 raft_consensus.cc:2229] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
W20260809 14:30:05.157887 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 422067edd08745bebdf59d4da704e665 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 422067edd08745bebdf59d4da704e665 already in progress: creating tablet
I20260809 14:30:05.158907 353 tablet_replica.cc:330] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:05.158890 426 raft_consensus.cc:2258] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.159786 353 raft_consensus.cc:2229] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:05.160385 353 raft_consensus.cc:2258] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.162025 426 ts_tablet_manager.cc:1905] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.162252 353 ts_tablet_manager.cc:1905] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.164479 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 119) took 2226 ms. Trace:
I20260809 14:30:05.164829 32763 rpcz_store.cc:276] 0809 14:30:02.984785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.987911 (+ 3126us) service_pool.cc:227] Handling call
0809 14:30:04.898869 (+1910958us) ts_tablet_manager.cc:1145] Deleting tablet 01cfc9e0319748c8b0e928fc64706933
0809 14:30:04.898928 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.043521 (+144593us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.102488 (+ 58967us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.164382 (+ 61894us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1910178}
I20260809 14:30:05.170154 32763 tablet_replica.cc:330] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:05.154247 32691 raft_consensus.cc:348] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:05.171515 32691 raft_consensus.cc:374] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:05.172984 32763 raft_consensus.cc:2229] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:05.172844 32691 raft_consensus.cc:729] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:05.180378 32763 raft_consensus.cc:2258] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.180941 32691 consensus_queue.cc:260] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:05.185130 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_057301f4ff6f475a8d7ee621860e31f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:05.166213 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 01cfc9e0319748c8b0e928fc64706933 (table default.Table_3b3fded95fa74c4086529b42834a0955 [id=f383923a235841a4a476b6cf31d33ef9]) successfully deleted
W20260809 14:30:05.193133 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_057301f4ff6f475a8d7ee621860e31f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:05.194269 32693 tablet_bootstrap.cc:492] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:05.194523 32691 ts_tablet_manager.cc:1428] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.042s user 0.002s sys 0.005s
I20260809 14:30:05.194677 32693 ts_tablet_manager.cc:1397] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.211s user 0.009s sys 0.000s
I20260809 14:30:05.196005 32763 ts_tablet_manager.cc:1905] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.201241 32693 raft_consensus.cc:348] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.202008 32693 raft_consensus.cc:374] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:05.204854 32691 tablet_bootstrap.cc:492] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:05.207899 32705 tablet_bootstrap.cc:492] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:05.208305 32705 ts_tablet_manager.cc:1397] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.119s user 0.010s sys 0.000s
I20260809 14:30:05.202250 32693 raft_consensus.cc:729] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:05.216739 32693 consensus_queue.cc:260] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.222425 32693 ts_tablet_manager.cc:1428] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.027s user 0.006s sys 0.001s
I20260809 14:30:05.223307 32693 tablet_bootstrap.cc:492] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:05.232074 32705 raft_consensus.cc:348] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.232758 32705 raft_consensus.cc:374] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:05.233022 32705 raft_consensus.cc:729] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:05.233616 32705 consensus_queue.cc:260] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.242868 32705 ts_tablet_manager.cc:1428] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.034s user 0.005s sys 0.000s
I20260809 14:30:05.268115 32705 tablet_bootstrap.cc:492] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:05.269701 32691 tablet_bootstrap.cc:654] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:05.276937 426 ts_tablet_manager.cc:1918] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:05.277611 426 log.cc:1196] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/8b9ae593e06b4e2a949b799380c1a3e3
I20260809 14:30:05.279109 426 ts_tablet_manager.cc:1939] T 8b9ae593e06b4e2a949b799380c1a3e3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:05.281997 353 ts_tablet_manager.cc:1918] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:05.285483 32693 tablet_bootstrap.cc:654] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:05.301674 31931 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_b80dbb38192f49f797ea56d783e01f17 [id=7e9823d249dd4c4ab3362be3b7a331b3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 67afcb47ffc947c5b1d98c96de333116 consensus state has no leader
I20260809 14:30:05.317057 32705 tablet_bootstrap.cc:654] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:05.329528 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 84c8eb8ad2dd406085779ed6a86eb70e
I20260809 14:30:05.331066 31942 catalog_manager.cc:6565] ReplaceTablet: tablet 84c8eb8ad2dd406085779ed6a86eb70e deleted and replaced by tablet c8a95e406071421cac1704a42a5f78ff
I20260809 14:30:05.332850 32763 ts_tablet_manager.cc:1918] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:05.333312 32763 log.cc:1196] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/27e2d8ea00d247f69c14e04728519f8b
I20260809 14:30:05.333317 32295 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c8eb8ad2dd406085779ed6a86eb70e with delete_type TABLET_DATA_DELETED (replaced by tablet c8a95e406071421cac1704a42a5f78ff) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:05.335166 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c8eb8ad2dd406085779ed6a86eb70e with delete_type TABLET_DATA_DELETED (replaced by tablet c8a95e406071421cac1704a42a5f78ff) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:05.334205 32763 ts_tablet_manager.cc:1939] T 27e2d8ea00d247f69c14e04728519f8b P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:05.338927 541 tablet.cc:1722] T c468382a28934d50babf7bbc9a834c08 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:05.346619 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_7a57afa3d1b54cefb7f0406668eeed73" } new_table_name: "default.Table_63e216f488874af6aeec9bc3f6053760" modify_external_catalogs: true
I20260809 14:30:05.352798 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_701ce3bbb495413c9497791f1a125753"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:05.355171 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_701ce3bbb495413c9497791f1a125753 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:05.287618 353 log.cc:1196] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/ce085227492e49549d3304a05a66a27d
I20260809 14:30:05.364380 353 ts_tablet_manager.cc:1939] T ce085227492e49549d3304a05a66a27d P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:05.330017 32691 tablet_bootstrap.cc:492] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:05.367172 32691 ts_tablet_manager.cc:1397] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.162s user 0.015s sys 0.001s
I20260809 14:30:05.368990 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 91) took 4381 ms. Trace:
I20260809 14:30:05.384783 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 121) took 2244 ms. Trace:
W20260809 14:30:05.385260 568 log.cc:924] Time spent T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260809 14:30:05.385154 32763 rpcz_store.cc:276] 0809 14:30:03.140236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.140402 (+ 166us) service_pool.cc:227] Handling call
0809 14:30:05.169420 (+2029018us) ts_tablet_manager.cc:1145] Deleting tablet 27e2d8ea00d247f69c14e04728519f8b
0809 14:30:05.169710 (+ 290us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.293113 (+123403us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.332547 (+ 39434us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.384717 (+ 52170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2028284}
I20260809 14:30:05.386533 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 128) took 1190 ms. Trace:
I20260809 14:30:05.386826 32763 rpcz_store.cc:276] 0809 14:30:04.198679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.199136 (+ 457us) service_pool.cc:227] Handling call
0809 14:30:05.386127 (+1186991us) ts_tablet_manager.cc:1145] Deleting tablet 609b3e6049ec4b0582d78037b7416710
0809 14:30:05.386182 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.386486 (+ 304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":949500}
I20260809 14:30:05.388020 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 130) took 1182 ms. Trace:
W20260809 14:30:05.389720 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2af8b48c8cde4f11a53eec939a1f8e3d has no consensus state
I20260809 14:30:05.393628 557 tablet.cc:1722] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:05.369447 32691 raft_consensus.cc:348] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.331975 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c8eb8ad2dd406085779ed6a86eb70e with delete_type TABLET_DATA_DELETED (replaced by tablet c8a95e406071421cac1704a42a5f78ff) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:05.395675 32691 raft_consensus.cc:374] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:05.395978 32691 raft_consensus.cc:729] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:05.396377 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 27e2d8ea00d247f69c14e04728519f8b (table default.Table_acb53b3bc5b24b989ccf44b01709c500 [id=4c072795da8241a18c26f3a01af4dda6]) successfully deleted
W20260809 14:30:05.397054 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 609b3e6049ec4b0582d78037b7416710 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 609b3e6049ec4b0582d78037b7416710 already in progress: creating tablet
I20260809 14:30:05.396692 32691 consensus_queue.cc:260] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.399245 32691 ts_tablet_manager.cc:1428] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260809 14:30:05.400064 32691 tablet_bootstrap.cc:492] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:05.400874 573 raft_consensus.cc:482] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:05.401571 573 raft_consensus.cc:504] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.408258 573 leader_election.cc:288] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:05.388299 32763 rpcz_store.cc:276] 0809 14:30:04.206030 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.210242 (+ 4212us) service_pool.cc:227] Handling call
0809 14:30:05.387578 (+1177336us) ts_tablet_manager.cc:1145] Deleting tablet 07ac3ef5573c40b894c9b2e743ca0582
0809 14:30:05.387626 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.387956 (+ 330us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":950772}
I20260809 14:30:05.413786 32763 tablet_replica.cc:330] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:05.414603 32763 raft_consensus.cc:2229] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:05.415131 32763 raft_consensus.cc:2258] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.422371 571 tablet.cc:1722] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:05.425330 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e846e795144e48e3b2a1eca8c1b26489" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:05.425928 31943 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 42) took 953 ms (client timeout 1000 ms). Trace:
W20260809 14:30:05.426308 31943 rpcz_store.cc:269] 0809 14:30:04.472840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.472961 (+ 121us) service_pool.cc:227] Handling call
0809 14:30:05.425847 (+952886us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260809 14:30:05.427414 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e846e795144e48e3b2a1eca8c1b26489" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:05.428282 32332 raft_consensus.cc:2454] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
W20260809 14:30:05.430085 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 07ac3ef5573c40b894c9b2e743ca0582 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07ac3ef5573c40b894c9b2e743ca0582 already in progress: creating tablet
I20260809 14:30:05.430004 32443 leader_election.cc:302] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:05.437628 573 raft_consensus.cc:2781] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:05.438288 573 raft_consensus.cc:482] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:05.438656 32295 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8b48c8cde4f11a53eec939a1f8e3d (DEFAULT_TABLE table=default.Table_63e216f488874af6aeec9bc3f6053760 [id=c6ab0416f2fd4793af541d3109ade09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.440097 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8b48c8cde4f11a53eec939a1f8e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.439152 573 raft_consensus.cc:3037] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:05.447438 426 rpcz_store.cc:276] 0809 14:30:00.988215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.993452 (+ 5237us) service_pool.cc:227] Handling call
0809 14:30:05.154257 (+4160805us) ts_tablet_manager.cc:1145] Deleting tablet 8b9ae593e06b4e2a949b799380c1a3e3
0809 14:30:05.154316 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.208940 (+ 54624us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.276581 (+ 67641us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.368929 (+ 92348us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4159753}
I20260809 14:30:05.449385 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 90) took 4462 ms. Trace:
I20260809 14:30:05.366662 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 85) took 4504 ms. Trace:
I20260809 14:30:05.449819 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4af0fdb719405fb8fefb029eb9c71d (DEFAULT_TABLE table=default.Table_f8b7cca659db4a67940b789660f281cf [id=8c958ed2c0404dd8aaaad86faeb284ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.451550 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4af0fdb719405fb8fefb029eb9c71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.450443 353 rpcz_store.cc:276] 0809 14:30:00.871951 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.935406 (+ 63455us) service_pool.cc:227] Handling call
0809 14:30:05.158735 (+4223329us) ts_tablet_manager.cc:1145] Deleting tablet ce085227492e49549d3304a05a66a27d
0809 14:30:05.158780 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.213732 (+ 54952us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.281578 (+ 67846us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.366591 (+ 85013us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4213973}
I20260809 14:30:05.453156 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 95) took 3043 ms. Trace:
I20260809 14:30:05.453779 353 rpcz_store.cc:276] 0809 14:30:02.478829 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.483189 (+ 4360us) service_pool.cc:227] Handling call
0809 14:30:05.452375 (+2969186us) ts_tablet_manager.cc:1145] Deleting tablet 2f171260cb764ac284aef85e4219767e
0809 14:30:05.452479 (+ 104us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.453060 (+ 581us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2966001}
I20260809 14:30:05.456274 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet ce085227492e49549d3304a05a66a27d (table default.Table_7b47ee94db13435cb8e6ea1c46c528f7 [id=1240bfe259ae4a2596755c9b3d3d7474]) successfully deleted
I20260809 14:30:05.426266 32167 raft_consensus.cc:2454] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:30:05.466928 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e8d440cfb3be48698670be598799bce1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:05.468937 32705 tablet_bootstrap.cc:492] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
W20260809 14:30:05.468957 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8d440cfb3be48698670be598799bce1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:05.469342 32705 ts_tablet_manager.cc:1397] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.201s user 0.015s sys 0.008s
I20260809 14:30:05.473132 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_7930652dac8a4ec885cd18e733b1e84e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:05.444307 32691 tablet_bootstrap.cc:654] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:05.480574 32763 ts_tablet_manager.cc:1905] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.489030 31939 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_82bc1bf6e0f04a4680201cc8e2704244" } new_table_name: "default.Table_5fdc5800b255423da5d44ad4ab4a1c2a" modify_external_catalogs: true
I20260809 14:30:05.492647 32705 raft_consensus.cc:348] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:05.493515 32705 raft_consensus.cc:374] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:05.493805 32705 raft_consensus.cc:729] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:05.494771 32705 consensus_queue.cc:260] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:05.498536 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8b48c8cde4f11a53eec939a1f8e3d (DEFAULT_TABLE table=default.Table_63e216f488874af6aeec9bc3f6053760 [id=c6ab0416f2fd4793af541d3109ade09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.500577 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8b48c8cde4f11a53eec939a1f8e3d (DEFAULT_TABLE table=default.Table_63e216f488874af6aeec9bc3f6053760 [id=c6ab0416f2fd4793af541d3109ade09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.503304 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8b48c8cde4f11a53eec939a1f8e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.503790 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 4cdfc56edcd34696a7b806b0ee34d594
I20260809 14:30:05.504967 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8b48c8cde4f11a53eec939a1f8e3d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:05.505340 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 2f171260cb764ac284aef85e4219767e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f171260cb764ac284aef85e4219767e already in progress: creating tablet
I20260809 14:30:05.507354 31944 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 189) took 1120 ms. Trace:
I20260809 14:30:05.507947 31944 rpcz_store.cc:276] 0809 14:30:04.387616 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.398520 (+ 10904us) service_pool.cc:227] Handling call
0809 14:30:04.400483 (+ 1963us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:04.401229 (+ 746us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:04.401382 (+ 153us) catalog_manager.cc:2761] Locking tablets
0809 14:30:05.409202 (+1007820us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:05.503336 (+ 94134us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:05.503572 (+ 236us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:05.503705 (+ 133us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:05.507300 (+ 3595us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:05.418717 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:05.419155 (+ 438us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:05.419192 (+ 37us) write_op.cc:435] Acquired schema lock
0809 14:30:05.419204 (+ 12us) tablet.cc:660] Decoding operations
0809 14:30:05.419577 (+ 373us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:05.419649 (+ 72us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:05.419670 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:05.420107 (+ 437us) tablet.cc:699] Row locks acquired
0809 14:30:05.420121 (+ 14us) write_op.cc:260] PREPARE: finished
0809 14:30:05.420227 (+ 106us) write_op.cc:270] Start()
0809 14:30:05.420304 (+ 77us) write_op.cc:276] Timestamp: P: 1786285805420167 usec, L: 0
0809 14:30:05.420319 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 14:30:05.420870 (+ 551us) log.cc:842] Serialized 731 byte log entry
0809 14:30:05.500053 (+ 79183us) op_driver.cc:444] REPLICATION: finished
0809 14:30:05.500982 (+ 929us) write_op.cc:301] APPLY: starting
0809 14:30:05.501122 (+ 140us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:05.501332 (+ 210us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:05.501563 (+ 231us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:05.501927 (+ 364us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:05.502234 (+ 307us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:05.502326 (+ 92us) write_op.cc:312] APPLY: finished
0809 14:30:05.502638 (+ 312us) log.cc:842] Serialized 29 byte log entry
0809 14:30:05.502860 (+ 222us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:05.502973 (+ 113us) write_op.cc:454] Released schema lock
0809 14:30:05.503030 (+ 57us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:05.503328 (+ 298us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:05.503705 (+ 377us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":698,"apply.run_cpu_time_us":2695,"apply.run_wall_time_us":3013,"num_ops":2,"prepare.queue_time_us":7294,"prepare.run_cpu_time_us":2380,"prepare.run_wall_time_us":2380,"replication_time_us":79611}]]}
I20260809 14:30:05.511907 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 97) took 2782 ms. Trace:
I20260809 14:30:05.517259 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cdfc56edcd34696a7b806b0ee34d594 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:05.515304 353 rpcz_store.cc:276] 0809 14:30:02.782945 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.785537 (+ 2592us) service_pool.cc:227] Handling call
0809 14:30:05.504492 (+2718955us) ts_tablet_manager.cc:1145] Deleting tablet b0ad3946f5de4387af3627a88b6404cf
0809 14:30:05.504573 (+ 81us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.511784 (+ 7211us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2648529}
I20260809 14:30:05.523566 353 tablet_replica.cc:330] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:05.525205 353 raft_consensus.cc:2229] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:05.526104 353 raft_consensus.cc:2258] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.473730 426 rpcz_store.cc:276] 0809 14:30:00.987320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.989237 (+ 1917us) service_pool.cc:227] Handling call
0809 14:30:05.448956 (+4459719us) ts_tablet_manager.cc:1145] Deleting tablet a9f61933c2004a43af2c748ea445942c
0809 14:30:05.449027 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.449299 (+ 272us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4451823}
I20260809 14:30:05.533777 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 92) took 4545 ms. Trace:
I20260809 14:30:05.534186 426 rpcz_store.cc:276] 0809 14:30:00.988937 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.990736 (+ 1799us) service_pool.cc:227] Handling call
0809 14:30:05.533458 (+4542722us) ts_tablet_manager.cc:1145] Deleting tablet 71aad93fcc8b42e6b55db6b3ed6b4f91
0809 14:30:05.533515 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.533713 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4493467}
I20260809 14:30:05.536017 426 tablet_replica.cc:330] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:05.537155 426 raft_consensus.cc:2229] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:05.537887 426 raft_consensus.cc:2258] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.486272 32693 tablet_bootstrap.cc:492] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:05.542474 32693 ts_tablet_manager.cc:1397] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.319s user 0.009s sys 0.007s
W20260809 14:30:05.499400 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7930652dac8a4ec885cd18e733b1e84e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:05.545989 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e2d8ea00d247f69c14e04728519f8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:05.547029 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4af0fdb719405fb8fefb029eb9c71d (DEFAULT_TABLE table=default.Table_f8b7cca659db4a67940b789660f281cf [id=8c958ed2c0404dd8aaaad86faeb284ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.548702 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4af0fdb719405fb8fefb029eb9c71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.549497 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_8df47c7eeba1459b8dceb9d0c3e49165"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:05.551061 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8df47c7eeba1459b8dceb9d0c3e49165 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:05.513039 31946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 56a7a1c8e38748cd9a3c4ccef2aea13c
I20260809 14:30:05.553663 573 raft_consensus.cc:504] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:05.515513 31937 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 37) took 1090 ms (client timeout 1000 ms). Trace:
W20260809 14:30:05.559991 31937 rpcz_store.cc:269] 0809 14:30:04.424906 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.425089 (+ 183us) service_pool.cc:227] Handling call
0809 14:30:05.515414 (+1090325us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20899,"reported_tablets":3}
W20260809 14:30:05.520953 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet b0ad3946f5de4387af3627a88b6404cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0ad3946f5de4387af3627a88b6404cf already in progress: creating tablet
I20260809 14:30:05.563999 32307 tablet_service.cc:1467] Processing CreateTablet for tablet ce9848ee8f7d45cba615b06b9c826afb (DEFAULT_TABLE table=default.Table_80c122d74d1d4ff598b1f8e97c766fcc [id=49782f8fd7b74990b01d83148e23af89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.565217 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce9848ee8f7d45cba615b06b9c826afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.569429 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:05.490520 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 8b9ae593e06b4e2a949b799380c1a3e3 (table default.Table_68743f87e3364e37a579888759048d44 [id=c52a212b949c4c93b9f6732bbcaaeb33]) successfully deleted
W20260809 14:30:05.570782 31930 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 80286.5us
I20260809 14:30:05.574681 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cdfc56edcd34696a7b806b0ee34d594 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:05.576292 32483 tablet_service.cc:1514] Processing DeleteTablet for tablet 969849d30c204937a7508e8d54dae427 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:55 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:05.576403 32481 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4af0fdb719405fb8fefb029eb9c71d (DEFAULT_TABLE table=default.Table_f8b7cca659db4a67940b789660f281cf [id=8c958ed2c0404dd8aaaad86faeb284ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.581732 353 ts_tablet_manager.cc:1905] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.586640 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e846e795144e48e3b2a1eca8c1b26489" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:05.587409 32167 raft_consensus.cc:3037] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:05.590878 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cdfc56edcd34696a7b806b0ee34d594 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:05.592613 32148 tablet_service.cc:1467] Processing CreateTablet for tablet ce9848ee8f7d45cba615b06b9c826afb (DEFAULT_TABLE table=default.Table_80c122d74d1d4ff598b1f8e97c766fcc [id=49782f8fd7b74990b01d83148e23af89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.593776 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce9848ee8f7d45cba615b06b9c826afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.594310 32705 ts_tablet_manager.cc:1428] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.125s user 0.003s sys 0.004s
I20260809 14:30:05.599891 31946 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_4be51cdd7c0f4caa9b2ba3719af75258" } new_table_name: "default.Table_0dcb148ef7df472db8e57a80264a25c2" modify_external_catalogs: true
I20260809 14:30:05.601256 573 leader_election.cc:288] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:05.602324 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e846e795144e48e3b2a1eca8c1b26489" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:05.602913 32332 raft_consensus.cc:3037] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:05.606648 32693 raft_consensus.cc:348] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.607450 32693 raft_consensus.cc:374] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:05.607729 32693 raft_consensus.cc:729] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:05.605850 32691 tablet_bootstrap.cc:492] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:05.610396 32691 ts_tablet_manager.cc:1397] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.210s user 0.014s sys 0.009s
I20260809 14:30:05.610575 32332 raft_consensus.cc:2454] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:05.611940 32443 leader_election.cc:302] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:05.613178 573 raft_consensus.cc:2781] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:05.603492 32167 raft_consensus.cc:2454] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:05.615914 32705 tablet_bootstrap.cc:492] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:05.616863 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4af0fdb719405fb8fefb029eb9c71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.614712 32691 raft_consensus.cc:348] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:05.620088 32691 raft_consensus.cc:374] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:05.620680 32691 raft_consensus.cc:729] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:05.622700 569 raft_consensus.cc:482] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:05.623087 569 raft_consensus.cc:504] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.628103 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 3d81046d929c48a3a6946f20d8fd7bca (DEFAULT_TABLE table=default.Table_971c53b113a249d28e56fd3f27219c60 [id=f60105c3fd98462b9367f400abca2c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.629988 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d81046d929c48a3a6946f20d8fd7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.608330 32693 consensus_queue.cc:260] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.638352 338 raft_consensus.cc:482] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:05.639189 338 raft_consensus.cc:504] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.640444 426 ts_tablet_manager.cc:1905] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.644817 573 raft_consensus.cc:686] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:05.654927 573 consensus_queue.cc:237] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.657910 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:05.658595 31944 master_service.cc:946] ReplaceTablet: received request to replace tablet 84c8eb8ad2dd406085779ed6a86eb70e from {username='slave'} at 127.0.0.1:49782
W20260809 14:30:05.662703 32260 leader_election.cc:341] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:05.639213 32693 ts_tablet_manager.cc:1428] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.096s user 0.002s sys 0.006s
I20260809 14:30:05.669682 32693 tablet_bootstrap.cc:492] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:05.665778 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:05.670482 32513 raft_consensus.cc:2454] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:05.672520 32483 tablet_service.cc:1467] Processing CreateTablet for tablet ce9848ee8f7d45cba615b06b9c826afb (DEFAULT_TABLE table=default.Table_80c122d74d1d4ff598b1f8e97c766fcc [id=49782f8fd7b74990b01d83148e23af89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.672740 32260 leader_election.cc:302] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:05.674583 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce9848ee8f7d45cba615b06b9c826afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.677580 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:05.687896 32763 ts_tablet_manager.cc:1918] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:05.687700 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 3d81046d929c48a3a6946f20d8fd7bca (DEFAULT_TABLE table=default.Table_971c53b113a249d28e56fd3f27219c60 [id=f60105c3fd98462b9367f400abca2c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.691280 32140 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:05.692257 450 raft_consensus.cc:482] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:05.692929 450 raft_consensus.cc:504] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.698030 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f36ab4503f24ca2a2a0d812c4c52468" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:05.700541 32439 leader_election.cc:341] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:05.704478 529 raft_consensus.cc:482] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:05.704993 529 raft_consensus.cc:504] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:05.708663 529 leader_election.cc:288] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:05.709820 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet a9f61933c2004a43af2c748ea445942c because the tablet was not found. No further retry: Not found: Tablet not found: a9f61933c2004a43af2c748ea445942c
I20260809 14:30:05.709901 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:05.710904 32513 raft_consensus.cc:2454] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:05.712401 32095 leader_election.cc:302] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:05.713687 529 raft_consensus.cc:2781] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:05.713587 450 leader_election.cc:288] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:05.714008 529 raft_consensus.cc:482] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:05.714294 529 raft_consensus.cc:3037] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:05.714771 32705 tablet_bootstrap.cc:654] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:05.714506 32140 tablet_service.cc:1467] Processing CreateTablet for tablet ea0a36a6ddc64557b31d0dfef8b5b457 (DEFAULT_TABLE table=default.Table_867a407e542b4a26af79c08ca5d0421f [id=58971413645d471181371b2d365e41b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.688308 32763 log.cc:1196] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/da7b2e6ca78b4fffb6f8daff1d874afe
I20260809 14:30:05.716998 32763 ts_tablet_manager.cc:1939] T da7b2e6ca78b4fffb6f8daff1d874afe P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:05.717123 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 3d81046d929c48a3a6946f20d8fd7bca (DEFAULT_TABLE table=default.Table_971c53b113a249d28e56fd3f27219c60 [id=f60105c3fd98462b9367f400abca2c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.718652 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d81046d929c48a3a6946f20d8fd7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.719497 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0a36a6ddc64557b31d0dfef8b5b457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.725267 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d81046d929c48a3a6946f20d8fd7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.755788 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 129) took 1550 ms. Trace:
I20260809 14:30:05.759136 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_057301f4ff6f475a8d7ee621860e31f1" } modify_external_catalogs: true
I20260809 14:30:05.759696 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_057301f4ff6f475a8d7ee621860e31f1" } modify_external_catalogs: true
I20260809 14:30:05.764394 426 ts_tablet_manager.cc:1918] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:05.765719 426 log.cc:1196] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/01cfc9e0319748c8b0e928fc64706933
I20260809 14:30:05.766885 426 ts_tablet_manager.cc:1939] T 01cfc9e0319748c8b0e928fc64706933 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:05.771523 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 93) took 4703 ms. Trace:
I20260809 14:30:05.776598 426 rpcz_store.cc:276] 0809 14:30:01.067940 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.068086 (+ 146us) service_pool.cc:227] Handling call
0809 14:30:05.535792 (+4467706us) ts_tablet_manager.cc:1145] Deleting tablet 01cfc9e0319748c8b0e928fc64706933
0809 14:30:05.535849 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.742136 (+206287us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.764118 (+ 21982us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.771465 (+ 7347us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4466928}
I20260809 14:30:05.767972 529 raft_consensus.cc:504] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:05.778185 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 01cfc9e0319748c8b0e928fc64706933 (table default.Table_3b3fded95fa74c4086529b42834a0955 [id=f383923a235841a4a476b6cf31d33ef9]) successfully deleted
I20260809 14:30:05.778549 426 tablet_replica.cc:330] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:05.779384 426 raft_consensus.cc:2229] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:05.780085 426 raft_consensus.cc:2258] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.756224 32763 rpcz_store.cc:276] 0809 14:30:04.205615 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.206685 (+ 1070us) service_pool.cc:227] Handling call
0809 14:30:05.413629 (+1206944us) ts_tablet_manager.cc:1145] Deleting tablet da7b2e6ca78b4fffb6f8daff1d874afe
0809 14:30:05.413679 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.603059 (+189380us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.687679 (+ 84620us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.755395 (+ 67716us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b5800001898. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c00ebf11 00007fb6c026f851 00007fb6c026fe82 00007fb6c912f8b9 00007fb6c81635d8 00007fb6c8227e42 00007fb6c8257e10 00007fb6c823ee0d 00007fb6c823edb9 00007fb6c823ed49 00007fb6c823ed11
0809 14:30:05.755712 (+ 317us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":80197888,"tablet-delete.queue_time_us":976657}
I20260809 14:30:05.785842 32763 tablet_replica.cc:330] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:05.786170 426 ts_tablet_manager.cc:1905] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.786522 32763 raft_consensus.cc:2229] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:05.786412 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet da7b2e6ca78b4fffb6f8daff1d874afe (table default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]) successfully deleted
I20260809 14:30:05.787137 32763 pending_rounds.cc:62] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:30:05.787344 32763 pending_rounds.cc:68] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626656814751744 op_type: NO_OP noop_request { }
I20260809 14:30:05.787794 32763 raft_consensus.cc:2866] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:05.788130 32763 raft_consensus.cc:2258] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.810694 426 ts_tablet_manager.cc:1918] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:05.811678 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet 4cdfc56edcd34696a7b806b0ee34d594 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:05.815265 426 log.cc:1196] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/27e2d8ea00d247f69c14e04728519f8b
I20260809 14:30:05.816372 426 ts_tablet_manager.cc:1939] T 27e2d8ea00d247f69c14e04728519f8b P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:05.822857 31938 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3e1ba4957ad24aa7adbabbab92339aa1" } modify_external_catalogs: true
I20260809 14:30:05.823451 31938 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3e1ba4957ad24aa7adbabbab92339aa1" } modify_external_catalogs: true
W20260809 14:30:05.742762 31939 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e28a765e1854486595683c0ac00da24c consensus state has no leader
I20260809 14:30:05.741575 569 leader_election.cc:288] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:05.838973 32432 log.cc:924] Time spent T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Append to log took a long time: real 0.080s user 0.001s sys 0.000s
I20260809 14:30:05.844143 32763 ts_tablet_manager.cc:1905] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.844609 529 leader_election.cc:288] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:05.845263 552 raft_consensus.cc:482] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:05.857148 552 raft_consensus.cc:504] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:05.857440 32482 tablet_service.cc:1467] Processing CreateTablet for tablet ea0a36a6ddc64557b31d0dfef8b5b457 (DEFAULT_TABLE table=default.Table_867a407e542b4a26af79c08ca5d0421f [id=58971413645d471181371b2d365e41b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.859328 552 leader_election.cc:288] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:05.859799 32763 ts_tablet_manager.cc:1918] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:05.860131 32763 log.cc:1196] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/ce085227492e49549d3304a05a66a27d
I20260809 14:30:05.861088 32763 ts_tablet_manager.cc:1939] T ce085227492e49549d3304a05a66a27d P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:05.863703 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 131) took 1556 ms. Trace:
I20260809 14:30:05.863629 32148 tablet_service.cc:1467] Processing CreateTablet for tablet faad36657cd24202abc0573c9e3aaa90 (DEFAULT_TABLE table=default.Table_6680c0954c634f329cc9a572853054d7 [id=77393de1074146b5ace396d3827d2003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.864087 32763 rpcz_store.cc:276] 0809 14:30:04.306989 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.319540 (+ 12551us) service_pool.cc:227] Handling call
0809 14:30:05.785641 (+1466101us) ts_tablet_manager.cc:1145] Deleting tablet ce085227492e49549d3304a05a66a27d
0809 14:30:05.785737 (+ 96us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.851954 (+ 66217us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.859605 (+ 7651us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.863630 (+ 4025us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1348529}
I20260809 14:30:05.864971 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faad36657cd24202abc0573c9e3aaa90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.865410 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet ce085227492e49549d3304a05a66a27d (table default.Table_7b47ee94db13435cb8e6ea1c46c528f7 [id=1240bfe259ae4a2596755c9b3d3d7474]) successfully deleted
I20260809 14:30:05.866273 32763 tablet_replica.cc:330] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:05.866997 32763 raft_consensus.cc:2229] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:05.867465 32763 raft_consensus.cc:2258] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.870143 32763 ts_tablet_manager.cc:1905] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.870580 32484 tablet_service.cc:1467] Processing CreateTablet for tablet b393017627934fa1b740d8ba8ac62d51 (DEFAULT_TABLE table=default.Table_8b32f662e5154cf78a133dd5249f5f0f [id=7043672a284145c7a5c9a5d80c322ef8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.871265 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:05.871821 32490 tablet_service.cc:1467] Processing CreateTablet for tablet faad36657cd24202abc0573c9e3aaa90 (DEFAULT_TABLE table=default.Table_6680c0954c634f329cc9a572853054d7 [id=77393de1074146b5ace396d3827d2003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.875267 32140 tablet_service.cc:1467] Processing CreateTablet for tablet b393017627934fa1b740d8ba8ac62d51 (DEFAULT_TABLE table=default.Table_8b32f662e5154cf78a133dd5249f5f0f [id=7043672a284145c7a5c9a5d80c322ef8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.881738 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b393017627934fa1b740d8ba8ac62d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.712227 338 leader_election.cc:288] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:05.855293 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_bbdad68122354be39969f430fdc7a0be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:05.885429 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bbdad68122354be39969f430fdc7a0be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:05.886919 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:05.883675 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f36ab4503f24ca2a2a0d812c4c52468" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:05.900748 579 raft_consensus.cc:2781] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:05.901249 579 raft_consensus.cc:482] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:05.901659 579 raft_consensus.cc:3037] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:05.910326 32443 leader_election.cc:341] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:05.911319 32443 leader_election.cc:302] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:05.887712 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 215263us
I20260809 14:30:05.920899 580 raft_consensus.cc:2726] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:05.901568 31946 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d9f01d25eac34d4d998dc457be6a5d9b consensus state has no leader
I20260809 14:30:05.923375 32485 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:05.923943 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 01cfc9e0319748c8b0e928fc64706933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:56 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:05.924659 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 422067edd08745bebdf59d4da704e665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:00 UTC) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:05.904713 31933 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.125:42169, user_credentials={real_user=slave}} blocked reactor thread for 63017.9us
I20260809 14:30:05.932545 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 422067edd08745bebdf59d4da704e665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:00 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:05.935026 32140 tablet_service.cc:1514] Processing DeleteTablet for tablet da7b2e6ca78b4fffb6f8daff1d874afe with delete_type TABLET_DATA_DELETED (replaced by tablet 37a3a71874fd44bf865b11c05e84ca84) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:05.941836 32691 consensus_queue.cc:260] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:05.949976 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:05.950935 32513 raft_consensus.cc:3037] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:05.952883 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422067edd08745bebdf59d4da704e665" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:05.953711 32514 raft_consensus.cc:2454] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:05.955471 32095 leader_election.cc:302] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:05.956162 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0a36a6ddc64557b31d0dfef8b5b457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.956423 529 raft_consensus.cc:2781] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:05.957046 529 raft_consensus.cc:482] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:05.957777 529 raft_consensus.cc:3037] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:05.957993 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 94) took 4876 ms. Trace:
I20260809 14:30:05.958380 426 rpcz_store.cc:276] 0809 14:30:01.081798 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.081978 (+ 180us) service_pool.cc:227] Handling call
0809 14:30:05.778349 (+4696371us) ts_tablet_manager.cc:1145] Deleting tablet 27e2d8ea00d247f69c14e04728519f8b
0809 14:30:05.778396 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.796446 (+ 18050us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.810451 (+ 14005us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:05.957922 (+147471us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4695698}
I20260809 14:30:05.959851 426 tablet_replica.cc:330] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:05.961889 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 27e2d8ea00d247f69c14e04728519f8b (table default.Table_acb53b3bc5b24b989ccf44b01709c500 [id=4c072795da8241a18c26f3a01af4dda6]) successfully deleted
I20260809 14:30:05.963830 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b393017627934fa1b740d8ba8ac62d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.966787 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faad36657cd24202abc0573c9e3aaa90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:05.968879 426 raft_consensus.cc:2229] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:05.969369 426 raft_consensus.cc:2258] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:05.971203 426 ts_tablet_manager.cc:1905] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:05.977572 32513 raft_consensus.cc:2454] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:05.993379 32299 tablet_service.cc:1467] Processing CreateTablet for tablet b393017627934fa1b740d8ba8ac62d51 (DEFAULT_TABLE table=default.Table_8b32f662e5154cf78a133dd5249f5f0f [id=7043672a284145c7a5c9a5d80c322ef8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:05.994565 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b393017627934fa1b740d8ba8ac62d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.005239 32307 tablet_service.cc:1467] Processing CreateTablet for tablet ea0a36a6ddc64557b31d0dfef8b5b457 (DEFAULT_TABLE table=default.Table_867a407e542b4a26af79c08ca5d0421f [id=58971413645d471181371b2d365e41b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.006572 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0a36a6ddc64557b31d0dfef8b5b457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.019222 529 raft_consensus.cc:504] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.020229 32705 tablet_bootstrap.cc:492] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:06.022063 32705 ts_tablet_manager.cc:1397] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.406s user 0.006s sys 0.034s
I20260809 14:30:06.025391 529 leader_election.cc:288] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:06.032548 32763 ts_tablet_manager.cc:1918] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:06.032966 32763 log.cc:1196] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/849d798165144531810a03dc2b9b5f1e
I20260809 14:30:06.033813 32763 ts_tablet_manager.cc:1939] T 849d798165144531810a03dc2b9b5f1e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:06.034504 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 132) took 1220 ms. Trace:
I20260809 14:30:06.034796 32763 rpcz_store.cc:276] 0809 14:30:04.814433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.825111 (+ 10678us) service_pool.cc:227] Handling call
0809 14:30:05.866130 (+1041019us) ts_tablet_manager.cc:1145] Deleting tablet 849d798165144531810a03dc2b9b5f1e
0809 14:30:05.866170 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.937242 (+ 71072us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.032243 (+ 95001us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.034438 (+ 2195us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1040060}
I20260809 14:30:06.035967 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]) successfully deleted
I20260809 14:30:06.046136 32300 tablet_service.cc:1467] Processing CreateTablet for tablet faad36657cd24202abc0573c9e3aaa90 (DEFAULT_TABLE table=default.Table_6680c0954c634f329cc9a572853054d7 [id=77393de1074146b5ace396d3827d2003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.053927 32763 tablet_replica.cc:330] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:06.053296 579 raft_consensus.cc:504] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.054994 32763 raft_consensus.cc:2229] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:06.055305 32331 raft_consensus.cc:2379] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6bed14aff53a41b2bbf5a13ce55dec5a in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:06.055615 32763 raft_consensus.cc:2258] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:06.057418 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9e998648ceee4a19835ede64b0ef1038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:06.059103 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e998648ceee4a19835ede64b0ef1038 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:06.060478 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 3619d49d0b944a91bcb181e884047b38 (DEFAULT_TABLE table=default.Table_52d55a3e0baf4604a7f9330e03b5b3eb [id=86cfe8ca68c848988241fc9398f85cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.062937 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3619d49d0b944a91bcb181e884047b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.063388 31937 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet e28a765e1854486595683c0ac00da24c (table default.Table_5fdc5800b255423da5d44ad4ab4a1c2a [id=20d2073f354649b3b4bc4d583660c93a]). Expected version 3 got 0
W20260809 14:30:06.063576 32260 consensus_peers.cc:487] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Couldn't send request to peer 6bed14aff53a41b2bbf5a13ce55dec5a. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:06.063927 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:06.065621 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:06.067114 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422067edd08745bebdf59d4da704e665" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:06.068960 32260 consensus_peers.cc:487] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:06.068538 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422067edd08745bebdf59d4da704e665" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
W20260809 14:30:06.071427 32260 consensus_peers.cc:487] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da7b2e6ca78b4fffb6f8daff1d874afe. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:06.073642 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faad36657cd24202abc0573c9e3aaa90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.083901 31946 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b0a7288bc1a843eabeb4ac3a50e99c45" } new_table_name: "default.Table_2c3e9ee14404475bba64491f9d132524" modify_external_catalogs: true
I20260809 14:30:06.073609 589 consensus_queue.cc:1035] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260809 14:30:06.089774 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3c7d49528e7746e7b4711c684fe4191e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:06.091209 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c7d49528e7746e7b4711c684fe4191e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:06.092990 579 leader_election.cc:288] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:06.096110 32099 leader_election.cc:341] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.097644 32693 tablet_bootstrap.cc:654] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:06.099037 32099 leader_election.cc:341] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:06.100315 32099 leader_election.cc:341] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:06.102005 32099 leader_election.cc:341] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.111791 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 3619d49d0b944a91bcb181e884047b38 (DEFAULT_TABLE table=default.Table_52d55a3e0baf4604a7f9330e03b5b3eb [id=86cfe8ca68c848988241fc9398f85cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.118093 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3619d49d0b944a91bcb181e884047b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.121590 31943 master_service.cc:946] ReplaceTablet: received request to replace tablet 07ac3ef5573c40b894c9b2e743ca0582 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:06.125406 31940 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_d1d760ad86f94839bf3b5b71dd8809aa [id=44f0605ad9e4418ca64ceca2a4120cf0] alter complete (version 1)
I20260809 14:30:06.133395 32693 tablet_bootstrap.cc:492] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:06.133817 32693 ts_tablet_manager.cc:1397] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.464s user 0.020s sys 0.018s
I20260809 14:30:06.134896 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9b2c82fab8874762a05ad054b98539d8" } new_table_name: "default.Table_d841d100269948ca8f7fc467c85a3b0d" modify_external_catalogs: true
I20260809 14:30:06.139763 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e2d8ea00d247f69c14e04728519f8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:57 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:06.149981 32693 raft_consensus.cc:348] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.150730 32693 raft_consensus.cc:374] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:06.151015 32693 raft_consensus.cc:729] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:06.151593 32693 consensus_queue.cc:260] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.155305 31937 catalog_manager.cc:5495] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 1 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:05.978384 580 raft_consensus.cc:482] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:30:05.982519 32439 leader_election.cc:341] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:06.162174 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 179683us
W20260809 14:30:06.025153 32443 consensus_peers.cc:487] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Couldn't send request to peer 3eb0bc287b12421d89abf97ca66e4d9b. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:06.163473 32482 tablet_service.cc:1467] Processing CreateTablet for tablet 3619d49d0b944a91bcb181e884047b38 (DEFAULT_TABLE table=default.Table_52d55a3e0baf4604a7f9330e03b5b3eb [id=86cfe8ca68c848988241fc9398f85cdf]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:06.163851 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 139004us
I20260809 14:30:06.164790 32443 leader_election.cc:302] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:06.127246 573 raft_consensus.cc:482] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:06.164786 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3619d49d0b944a91bcb181e884047b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.165777 573 raft_consensus.cc:504] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:06.168823 450 raft_consensus.cc:3037] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:06.169860 32693 ts_tablet_manager.cc:1428] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.036s user 0.007s sys 0.000s
I20260809 14:30:06.162176 580 raft_consensus.cc:504] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:06.174292 32763 ts_tablet_manager.cc:1905] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:06.175743 580 leader_election.cc:288] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:06.180509 32095 leader_election.cc:302] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:30:06.185913 353 ts_tablet_manager.cc:1918] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:06.186321 353 log.cc:1196] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/bfb1dd7b6b6243b4ba3eee9b1b3b1f1c
I20260809 14:30:06.186560 32693 tablet_bootstrap.cc:492] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:06.187106 353 ts_tablet_manager.cc:1939] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:06.187638 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:06.190362 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609b3e6049ec4b0582d78037b7416710" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:06.168179 573 leader_election.cc:288] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:06.197247 32705 raft_consensus.cc:348] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.198074 32705 raft_consensus.cc:374] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:06.198326 32705 raft_consensus.cc:729] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:06.198915 32705 consensus_queue.cc:260] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.204192 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:06.204584 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609b3e6049ec4b0582d78037b7416710" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:06.205639 32443 leader_election.cc:341] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:06.206349 32443 leader_election.cc:341] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.202756 32763 ts_tablet_manager.cc:1918] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260809 14:30:06.210559 32439 leader_election.cc:341] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.211007 32763 log.cc:1196] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/e378d7763c5046249011a6d6e0617c97
I20260809 14:30:06.211140 32439 leader_election.cc:302] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:06.211830 32763 ts_tablet_manager.cc:1939] T e378d7763c5046249011a6d6e0617c97 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
W20260809 14:30:06.212572 32439 leader_election.cc:341] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.212868 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 133) took 1398 ms. Trace:
I20260809 14:30:06.213045 32439 leader_election.cc:302] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:06.213388 32763 rpcz_store.cc:276] 0809 14:30:04.815029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.826306 (+ 11277us) service_pool.cc:227] Handling call
0809 14:30:06.053732 (+1227426us) ts_tablet_manager.cc:1145] Deleting tablet e378d7763c5046249011a6d6e0617c97
0809 14:30:06.053791 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:06.183999 (+130208us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.202532 (+ 18533us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.212800 (+ 10268us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":116459,"tablet-delete.queue_time_us":1226760}
I20260809 14:30:06.214356 569 raft_consensus.cc:2726] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:06.214854 569 raft_consensus.cc:2726] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:06.215070 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet e378d7763c5046249011a6d6e0617c97 (table default.Table_3ecd77174e9948a59988f0b186dc8793 [id=82601d705044402cbf2820f18668da15]) successfully deleted
I20260809 14:30:06.218834 32763 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 134) took 1403 ms. Trace:
I20260809 14:30:06.219136 32763 rpcz_store.cc:276] 0809 14:30:04.815532 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.827424 (+ 11892us) service_pool.cc:227] Handling call
0809 14:30:06.218506 (+1391082us) ts_tablet_manager.cc:1145] Deleting tablet faf222c49c934d52a2642f20e8575702
0809 14:30:06.218548 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:06.218784 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1390246}
W20260809 14:30:06.220538 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet faf222c49c934d52a2642f20e8575702 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet faf222c49c934d52a2642f20e8575702 already in progress: creating tablet
I20260809 14:30:06.221103 450 raft_consensus.cc:2726] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:06.222779 32763 tablet_replica.cc:330] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:06.224123 31945 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet cbdcd638100f4d6caf23dc5b53b43b30
I20260809 14:30:06.225433 32763 raft_consensus.cc:2229] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:06.230587 32693 tablet_bootstrap.cc:654] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:06.232348 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 99) took 3354 ms. Trace:
I20260809 14:30:06.232694 353 rpcz_store.cc:276] 0809 14:30:02.886003 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.891556 (+ 5553us) service_pool.cc:227] Handling call
0809 14:30:05.518025 (+2626469us) ts_tablet_manager.cc:1145] Deleting tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c
0809 14:30:05.518092 (+ 67us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:05.874628 (+356536us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.158258 (+283630us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.232287 (+ 74029us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2625490}
I20260809 14:30:06.233804 353 tablet_replica.cc:330] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:06.234014 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c (table default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b]) successfully deleted
I20260809 14:30:06.234447 353 raft_consensus.cc:2229] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:06.234929 353 raft_consensus.cc:2258] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:06.240746 552 raft_consensus.cc:2781] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:06.240861 32763 raft_consensus.cc:2258] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:06.243168 552 raft_consensus.cc:686] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:30:06.243446 32763 ts_tablet_manager.cc:1905] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:06.244392 552 consensus_queue.cc:237] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:06.147541 32691 ts_tablet_manager.cc:1428] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.537s user 0.005s sys 0.002s
I20260809 14:30:06.148355 426 ts_tablet_manager.cc:1918] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:06.247946 426 log.cc:1196] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/e378d7763c5046249011a6d6e0617c97
I20260809 14:30:06.248687 426 ts_tablet_manager.cc:1939] T e378d7763c5046249011a6d6e0617c97 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:06.250653 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 95) took 5166 ms. Trace:
I20260809 14:30:06.250929 426 rpcz_store.cc:276] 0809 14:30:01.083995 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.084142 (+ 147us) service_pool.cc:227] Handling call
0809 14:30:05.959675 (+4875533us) ts_tablet_manager.cc:1145] Deleting tablet e378d7763c5046249011a6d6e0617c97
0809 14:30:05.959728 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:06.084370 (+124642us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.148123 (+ 63753us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.250594 (+102471us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4874838}
I20260809 14:30:06.252002 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet e378d7763c5046249011a6d6e0617c97 (table default.Table_3ecd77174e9948a59988f0b186dc8793 [id=82601d705044402cbf2820f18668da15]) successfully deleted
I20260809 14:30:06.253240 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 103) took 4464 ms. Trace:
I20260809 14:30:06.253561 426 rpcz_store.cc:276] 0809 14:30:01.789178 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.789382 (+ 204us) service_pool.cc:227] Handling call
0809 14:30:06.252928 (+4463546us) ts_tablet_manager.cc:1145] Deleting tablet 2f171260cb764ac284aef85e4219767e
0809 14:30:06.252965 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:06.253194 (+ 229us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4462956}
W20260809 14:30:06.254673 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 2f171260cb764ac284aef85e4219767e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f171260cb764ac284aef85e4219767e already in progress: creating tablet
I20260809 14:30:06.159667 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422067edd08745bebdf59d4da704e665" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:06.255403 32514 raft_consensus.cc:3037] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:06.259646 32693 tablet_bootstrap.cc:492] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:06.260083 32693 ts_tablet_manager.cc:1397] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.005s
I20260809 14:30:06.263479 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 105) took 4260 ms. Trace:
I20260809 14:30:06.263852 426 rpcz_store.cc:276] 0809 14:30:02.022047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.022239 (+ 192us) service_pool.cc:227] Handling call
0809 14:30:06.263114 (+4240875us) ts_tablet_manager.cc:1145] Deleting tablet b0ad3946f5de4387af3627a88b6404cf
0809 14:30:06.263152 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:06.263430 (+ 278us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4240033}
W20260809 14:30:06.265143 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet b0ad3946f5de4387af3627a88b6404cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0ad3946f5de4387af3627a88b6404cf already in progress: creating tablet
I20260809 14:30:06.265913 426 tablet_replica.cc:330] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:06.266577 426 raft_consensus.cc:2229] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:06.267500 426 raft_consensus.cc:2258] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:06.267684 32693 raft_consensus.cc:348] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.269433 426 ts_tablet_manager.cc:1905] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:06.270151 32691 ts_tablet_manager.cc:1460] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a: Tablet startup took 659ms
I20260809 14:30:06.274322 32763 ts_tablet_manager.cc:1918] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:06.274701 32763 log.cc:1196] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/84c8eb8ad2dd406085779ed6a86eb70e
W20260809 14:30:06.273775 32691 ts_tablet_manager.cc:1462] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a: Trace:
0809 14:30:00.430400 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:00.430665 (+ 265us) service_pool.cc:227] Handling call
0809 14:30:00.432533 (+ 1868us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:00.432663 (+ 130us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:00.492813 (+ 60150us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:00.520008 (+ 27195us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:00.520506 (+ 498us) inbound_call.cc:173] Queueing success response
0809 14:30:05.399821 (+4879315us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 14:30:05.610667 (+210846us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 14:30:05.611438 (+ 771us) tablet_replica.cc:259] Starting instrumentation
0809 14:30:05.611864 (+ 426us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":1701,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":955,"tablet-open.queue_time_us":4879485}
I20260809 14:30:06.275585 32763 ts_tablet_manager.cc:1939] T 84c8eb8ad2dd406085779ed6a86eb70e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:06.268360 32693 raft_consensus.cc:374] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:06.276320 32693 raft_consensus.cc:729] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:06.277170 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 84c8eb8ad2dd406085779ed6a86eb70e (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
I20260809 14:30:06.276968 32693 consensus_queue.cc:260] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:06.278812 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 27e2d8ea00d247f69c14e04728519f8b because the tablet was not found. No further retry: Not found: Tablet not found: 27e2d8ea00d247f69c14e04728519f8b
W20260809 14:30:06.285730 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 4cdfc56edcd34696a7b806b0ee34d594 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cdfc56edcd34696a7b806b0ee34d594 already in progress: creating tablet
I20260809 14:30:06.289050 32763 tablet_replica.cc:330] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:06.289250 32514 raft_consensus.cc:2454] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:06.290313 32763 raft_consensus.cc:2229] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:06.291553 32763 raft_consensus.cc:2258] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:06.292968 32705 ts_tablet_manager.cc:1428] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.271s user 0.007s sys 0.001s
I20260809 14:30:06.290546 32095 leader_election.cc:302] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
W20260809 14:30:06.294162 32095 raft_consensus.cc:2670] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:06.295235 32763 ts_tablet_manager.cc:1905] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:06.302686 31931 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_b80dbb38192f49f797ea56d783e01f17 [id=7e9823d249dd4c4ab3362be3b7a331b3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 67afcb47ffc947c5b1d98c96de333116 consensus state has no leader
I20260809 14:30:06.309751 32693 ts_tablet_manager.cc:1428] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.049s user 0.008s sys 0.000s
I20260809 14:30:06.313189 31939 master_service.cc:946] ReplaceTablet: received request to replace tablet 79bb4dd74afe4ace94eb1519950a83c0 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:06.317351 32691 tablet_bootstrap.cc:492] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:06.335922 32705 tablet_bootstrap.cc:492] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:06.346997 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_533f32539e9542569c81b5bfe1f0f53e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:06.348531 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_533f32539e9542569c81b5bfe1f0f53e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:06.349778 31938 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 9e8406ce042d4e5dbf9f855eb20ef620
I20260809 14:30:06.356804 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:06.359439 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:06.361224 32705 tablet_bootstrap.cc:654] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:06.363029 32763 ts_tablet_manager.cc:1918] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:06.364199 32763 log.cc:1196] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/422067edd08745bebdf59d4da704e665
I20260809 14:30:06.365000 32763 ts_tablet_manager.cc:1939] T 422067edd08745bebdf59d4da704e665 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:06.380903 32513 raft_consensus.cc:2454] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:06.383291 31938 master_service.cc:946] ReplaceTablet: received request to replace tablet 3d592b1c4520467791ffd9d36e0d97fd from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:06.392649 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 7856076c1332479a894d2d69d16b19f5 (DEFAULT_TABLE table=default.Table_05c045fe502848f4805ad20c9362876f [id=7b7065fb14794d6aa1ca0e92818109d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.394245 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 7856076c1332479a894d2d69d16b19f5 (DEFAULT_TABLE table=default.Table_05c045fe502848f4805ad20c9362876f [id=7b7065fb14794d6aa1ca0e92818109d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.394783 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7856076c1332479a894d2d69d16b19f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.395821 353 ts_tablet_manager.cc:1905] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:06.402055 32693 tablet_bootstrap.cc:492] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:06.405943 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7856076c1332479a894d2d69d16b19f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.430764 32705 tablet_bootstrap.cc:492] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:06.442441 32705 ts_tablet_manager.cc:1397] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.149s user 0.010s sys 0.006s
I20260809 14:30:06.446450 32705 raft_consensus.cc:348] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:06.449975 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56a7a1c8e38748cd9a3c4ccef2aea13c already in progress: creating tablet
I20260809 14:30:06.450675 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 422067edd08745bebdf59d4da704e665 (table default.Table_d200d3ada57d40339da8cd5f8499cabc [id=8d157c6db35a4a6f976fed683f2d1d17]) successfully deleted
W20260809 14:30:06.451300 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet da7b2e6ca78b4fffb6f8daff1d874afe because the tablet was not found. No further retry: Not found: Tablet not found: da7b2e6ca78b4fffb6f8daff1d874afe
I20260809 14:30:06.457263 32705 raft_consensus.cc:374] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:06.458390 32705 raft_consensus.cc:729] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:06.459206 32705 consensus_queue.cc:260] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.469458 32693 tablet_bootstrap.cc:654] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:06.469467 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_63e216f488874af6aeec9bc3f6053760" } modify_external_catalogs: true
I20260809 14:30:06.470197 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_63e216f488874af6aeec9bc3f6053760" } modify_external_catalogs: true
W20260809 14:30:06.474392 32260 consensus_peers.cc:487] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e378d7763c5046249011a6d6e0617c97. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:06.475128 32260 leader_election.cc:341] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:06.475914 32260 consensus_peers.cc:487] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 84c8eb8ad2dd406085779ed6a86eb70e. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:06.476596 32260 consensus_peers.cc:487] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Couldn't send request to peer 6bed14aff53a41b2bbf5a13ce55dec5a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01cfc9e0319748c8b0e928fc64706933. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:06.477450 32705 ts_tablet_manager.cc:1428] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.029s user 0.003s sys 0.003s
I20260809 14:30:06.473632 31939 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
I20260809 14:30:06.479317 31939 catalog_manager.cc:5495] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65). New cstate: current_term: 1 leader_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:06.481033 31939 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet e28a765e1854486595683c0ac00da24c (table default.Table_5fdc5800b255423da5d44ad4ab4a1c2a [id=20d2073f354649b3b4bc4d583660c93a]). Expected version 3 got 0
I20260809 14:30:06.484181 32260 leader_election.cc:302] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:06.486495 32705 tablet_bootstrap.cc:492] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:06.486626 552 raft_consensus.cc:482] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:06.487664 589 raft_consensus.cc:2781] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:06.487372 552 raft_consensus.cc:504] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.488325 589 raft_consensus.cc:686] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:06.489881 589 consensus_queue.cc:237] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.493018 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a7a1c8e38748cd9a3c4ccef2aea13c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:06.493707 32513 raft_consensus.cc:2454] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:06.494863 32095 leader_election.cc:302] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:06.495774 593 raft_consensus.cc:2781] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:06.496419 593 raft_consensus.cc:482] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:06.497057 593 raft_consensus.cc:3037] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:06.498543 552 leader_election.cc:288] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:06.499569 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a7a1c8e38748cd9a3c4ccef2aea13c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:06.508046 32099 leader_election.cc:341] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.513973 32299 tablet_service.cc:1467] Processing CreateTablet for tablet d4980140ba2a4ecc87001e2a43c8e813 (DEFAULT_TABLE table=default.Table_5b7c498d7ea14448a8adbb89112bd384 [id=5ec6419d2680485bbb374fd175020d3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.530073 32301 tablet_service.cc:1467] Processing CreateTablet for tablet a5634154c6f149d3b2d36b961aee38e0 (DEFAULT_TABLE table=default.Table_f00ff0058556436f9c41b2b6532452b1 [id=52214a7198564a2abdde0e9a2e692ae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.515225 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4980140ba2a4ecc87001e2a43c8e813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.535457 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5634154c6f149d3b2d36b961aee38e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.531332 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 6fa25dbd78f64e5db9e31d433e8f4736 (DEFAULT_TABLE table=default.Table_76064ebb420d48dfb1586d58bf81d6a3 [id=9a90fee262334c4881f9202da51858ce]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:06.567329 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 83218.3us
I20260809 14:30:06.568148 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 7856076c1332479a894d2d69d16b19f5 (DEFAULT_TABLE table=default.Table_05c045fe502848f4805ad20c9362876f [id=7b7065fb14794d6aa1ca0e92818109d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.568998 589 raft_consensus.cc:482] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bed14aff53a41b2bbf5a13ce55dec5a)
I20260809 14:30:06.569674 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7856076c1332479a894d2d69d16b19f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.569650 589 raft_consensus.cc:504] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.573256 589 leader_election.cc:288] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:06.579387 32691 tablet_bootstrap.cc:654] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:06.581723 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fa25dbd78f64e5db9e31d433e8f4736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.582427 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 6fa25dbd78f64e5db9e31d433e8f4736 (DEFAULT_TABLE table=default.Table_76064ebb420d48dfb1586d58bf81d6a3 [id=9a90fee262334c4881f9202da51858ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.583698 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fa25dbd78f64e5db9e31d433e8f4736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.598311 579 raft_consensus.cc:482] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:06.598543 32481 tablet_service.cc:1467] Processing CreateTablet for tablet d4980140ba2a4ecc87001e2a43c8e813 (DEFAULT_TABLE table=default.Table_5b7c498d7ea14448a8adbb89112bd384 [id=5ec6419d2680485bbb374fd175020d3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.598866 579 raft_consensus.cc:504] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.601501 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 6fa25dbd78f64e5db9e31d433e8f4736 (DEFAULT_TABLE table=default.Table_76064ebb420d48dfb1586d58bf81d6a3 [id=9a90fee262334c4881f9202da51858ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.601977 32140 tablet_service.cc:1467] Processing CreateTablet for tablet d4980140ba2a4ecc87001e2a43c8e813 (DEFAULT_TABLE table=default.Table_5b7c498d7ea14448a8adbb89112bd384 [id=5ec6419d2680485bbb374fd175020d3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.602663 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fa25dbd78f64e5db9e31d433e8f4736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.603446 32705 tablet_bootstrap.cc:654] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:06.603855 450 consensus_queue.cc:1035] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:06.607904 579 leader_election.cc:288] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:06.610246 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4980140ba2a4ecc87001e2a43c8e813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.618506 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4980140ba2a4ecc87001e2a43c8e813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.622474 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8792541442324807881089a25803c23e" } modify_external_catalogs: true
I20260809 14:30:06.627934 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3705cf965d214eb794984b9c16604326" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:06.634892 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3705cf965d214eb794984b9c16604326" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:06.635520 32513 raft_consensus.cc:2454] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:06.637826 32148 tablet_service.cc:1467] Processing CreateTablet for tablet a5634154c6f149d3b2d36b961aee38e0 (DEFAULT_TABLE table=default.Table_f00ff0058556436f9c41b2b6532452b1 [id=52214a7198564a2abdde0e9a2e692ae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.638857 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5634154c6f149d3b2d36b961aee38e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.662039 32693 tablet_bootstrap.cc:492] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:06.662457 32693 ts_tablet_manager.cc:1397] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.261s user 0.010s sys 0.008s
W20260809 14:30:06.656577 32260 leader_election.cc:341] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Not found: Tablet not found: 13f6c6a2ff894ec3a795f0c3f044ec17
W20260809 14:30:06.668956 32260 leader_election.cc:341] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:06.669550 32260 leader_election.cc:341] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 13f6c6a2ff894ec3a795f0c3f044ec17
I20260809 14:30:06.581139 426 ts_tablet_manager.cc:1918] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:06.669854 32260 leader_election.cc:302] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:06.678818 426 log.cc:1196] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/bfb1dd7b6b6243b4ba3eee9b1b3b1f1c
I20260809 14:30:06.684408 579 raft_consensus.cc:2726] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:06.684837 32260 leader_election.cc:302] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:06.685532 579 raft_consensus.cc:2781] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:06.685866 579 raft_consensus.cc:482] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:06.686287 579 raft_consensus.cc:3037] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:06.705583 450 raft_consensus.cc:482] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:06.706796 450 raft_consensus.cc:504] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.710960 32483 tablet_service.cc:1467] Processing CreateTablet for tablet a5634154c6f149d3b2d36b961aee38e0 (DEFAULT_TABLE table=default.Table_f00ff0058556436f9c41b2b6532452b1 [id=52214a7198564a2abdde0e9a2e692ae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:06.716167 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:06.724833 552 raft_consensus.cc:482] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:30:06.721535 32443 leader_election.cc:341] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.725759 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5634154c6f149d3b2d36b961aee38e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:06.725404 552 raft_consensus.cc:504] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.727434 552 leader_election.cc:288] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:06.717461 450 leader_election.cc:288] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:06.529711 593 raft_consensus.cc:504] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.731714 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:06.732347 32513 raft_consensus.cc:2454] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:06.733949 32095 leader_election.cc:302] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:06.734472 426 ts_tablet_manager.cc:1939] T bfb1dd7b6b6243b4ba3eee9b1b3b1f1c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:06.742008 32705 tablet_bootstrap.cc:492] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:06.742660 32705 ts_tablet_manager.cc:1397] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.256s user 0.010s sys 0.011s
I20260809 14:30:06.744716 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:06.747567 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:06.750651 32099 leader_election.cc:341] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.749951 552 raft_consensus.cc:2781] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:06.751710 552 raft_consensus.cc:482] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:06.752175 552 raft_consensus.cc:3037] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:06.762331 32705 raft_consensus.cc:348] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.763564 32705 raft_consensus.cc:374] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:06.763942 32705 raft_consensus.cc:729] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:06.765245 32705 consensus_queue.cc:260] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.789850 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a7a1c8e38748cd9a3c4ccef2aea13c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:06.791738 31944 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:49782 (request call id 211) took 1094 ms. Trace:
W20260809 14:30:06.791769 32099 leader_election.cc:341] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.795596 32691 tablet_bootstrap.cc:492] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:06.796022 32691 ts_tablet_manager.cc:1397] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.479s user 0.005s sys 0.013s
I20260809 14:30:06.792120 31944 rpcz_store.cc:276] 0809 14:30:05.697373 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.700110 (+ 2737us) service_pool.cc:227] Handling call
0809 14:30:05.855077 (+154967us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9156 00007ffa4b016034 00007ffa4b015fc2 00007ffa4b015ef1 00007ffa4b015e6c 00007ffa4b015122 00007ffa492b894c 00007ffa492b7d86
0809 14:30:05.922873 (+ 67796us) catalog_manager.cc:2075] Acquired catalog manager lock
0809 14:30:06.019202 (+ 96329us) catalog_manager.cc:2121] Created new table and tablet info
0809 14:30:06.590335 (+571133us) catalog_manager.cc:2181] Wrote table and tablets to system table
0809 14:30:06.791391 (+201056us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0809 14:30:06.791660 (+ 269us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:06.394340 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:06.394736 (+ 396us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:06.394761 (+ 25us) write_op.cc:435] Acquired schema lock
0809 14:30:06.394774 (+ 13us) tablet.cc:660] Decoding operations
0809 14:30:06.395089 (+ 315us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:06.395128 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:06.395160 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:06.395381 (+ 221us) tablet.cc:699] Row locks acquired
0809 14:30:06.395394 (+ 13us) write_op.cc:260] PREPARE: finished
0809 14:30:06.449469 (+ 54075us) write_op.cc:270] Start()
0809 14:30:06.500104 (+ 50635us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b5400011728. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4d9a5e91 00007ffa4d9a4afc 00007ffa4b97b304 00007ffa4b9667e3 00007ffa4b9662b8 00007ffa4b96aba1 00007ffa4b96ab59 00007ffa4b96aae9 00007ffa4b96aab1 00007ffa4b969dad 0000000000580784
0809 14:30:06.500444 (+ 340us) write_op.cc:276] Timestamp: P: 1786285806447889 usec, L: 0
0809 14:30:06.500512 (+ 68us) op_driver.cc:331] REPLICATION: starting
0809 14:30:06.501193 (+ 681us) log.cc:842] Serialized 433 byte log entry
0809 14:30:06.513975 (+ 12782us) op_driver.cc:444] REPLICATION: finished
0809 14:30:06.543691 (+ 29716us) write_op.cc:301] APPLY: starting
0809 14:30:06.543809 (+ 118us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:06.544024 (+ 215us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:06.544082 (+ 58us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:06.545719 (+ 1637us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:06.546070 (+ 351us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 14:30:06.546174 (+ 104us) write_op.cc:312] APPLY: finished
0809 14:30:06.546495 (+ 321us) log.cc:842] Serialized 29 byte log entry
0809 14:30:06.589856 (+ 43361us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:06.589985 (+ 129us) write_op.cc:454] Released schema lock
0809 14:30:06.590029 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:06.591216 (+ 1187us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:06.591292 (+ 76us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":94997760,"child_traces":[["op",{"apply.queue_time_us":29402,"apply.run_cpu_time_us":3643,"apply.run_wall_time_us":47971,"num_ops":2,"prepare.queue_time_us":312332,"prepare.run_cpu_time_us":4051,"prepare.run_wall_time_us":107854,"raft.queue_time_us":82,"raft.run_cpu_time_us":554,"raft.run_wall_time_us":554,"replication_time_us":13353,"spinlock_wait_cycles":96714112}]]}
W20260809 14:30:06.797284 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2a689385d063469595fcae82aa979bb1 consensus state has no leader
W20260809 14:30:06.797981 31946 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 03f6a524d4ef48c4a31e0bfec2f557bb has no consensus state
I20260809 14:30:06.798362 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8792541442324807881089a25803c23e" } modify_external_catalogs: true
I20260809 14:30:06.802510 593 leader_election.cc:288] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:06.808434 32705 ts_tablet_manager.cc:1428] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.065s user 0.007s sys 0.005s
I20260809 14:30:06.809759 32705 tablet_bootstrap.cc:492] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:06.814568 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_753e14e7053644a3a98b60a13a6c9a66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:06.815047 32300 tablet_service.cc:1514] Processing DeleteTablet for tablet 422067edd08745bebdf59d4da704e665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:00 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:06.816234 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_753e14e7053644a3a98b60a13a6c9a66 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:06.820832 552 raft_consensus.cc:504] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.846316 615 consensus_queue.cc:1035] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:06.851130 32169 raft_consensus.cc:2379] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6bed14aff53a41b2bbf5a13ce55dec5a in current term 1: Already voted for candidate e1f94e0d792c46c9a0b6ab42820da6eb in this term.
I20260809 14:30:06.858978 353 ts_tablet_manager.cc:1918] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:06.862080 32439 leader_election.cc:302] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:06.864620 617 raft_consensus.cc:3037] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:06.865257 552 leader_election.cc:288] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:06.879920 617 raft_consensus.cc:2726] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:06.885967 32705 tablet_bootstrap.cc:654] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:06.868079 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:06.887080 353 log.cc:1196] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/8b9ae593e06b4e2a949b799380c1a3e3
I20260809 14:30:06.883775 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a7a1c8e38748cd9a3c4ccef2aea13c" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:06.885601 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:06.889328 32513 raft_consensus.cc:3037] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:06.889986 353 ts_tablet_manager.cc:1939] T 8b9ae593e06b4e2a949b799380c1a3e3 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:06.890988 32693 raft_consensus.cc:348] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:06.891748 32693 raft_consensus.cc:374] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:06.891973 32693 raft_consensus.cc:729] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:06.892025 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 101) took 3989 ms. Trace:
W20260809 14:30:06.888442 32099 leader_election.cc:341] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:06.913475 32513 raft_consensus.cc:2454] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:06.915606 32095 leader_election.cc:302] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:30:06.917086 552 raft_consensus.cc:2781] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:06.835974 579 raft_consensus.cc:504] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.923382 579 leader_election.cc:288] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:06.922538 31937 catalog_manager.cc:5495] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } }
I20260809 14:30:06.927065 31938 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 3d592b1c4520467791ffd9d36e0d97fd
I20260809 14:30:06.918010 552 raft_consensus.cc:686] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:30:06.932686 32691 raft_consensus.cc:348] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:06.933429 32691 raft_consensus.cc:374] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:06.933157 32512 raft_consensus.cc:2454] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:06.892306 353 rpcz_store.cc:276] 0809 14:30:02.902522 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.902702 (+ 180us) service_pool.cc:227] Handling call
0809 14:30:06.233668 (+3330966us) ts_tablet_manager.cc:1145] Deleting tablet 8b9ae593e06b4e2a949b799380c1a3e3
0809 14:30:06.233707 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:06.443414 (+209707us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.858716 (+415302us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.891961 (+ 33245us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3330227}
I20260809 14:30:06.934947 353 tablet_replica.cc:330] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:06.935060 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 8b9ae593e06b4e2a949b799380c1a3e3 (table default.Table_68743f87e3364e37a579888759048d44 [id=c52a212b949c4c93b9f6732bbcaaeb33]) successfully deleted
I20260809 14:30:06.938652 32095 leader_election.cc:302] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:30:06.940197 353 raft_consensus.cc:2229] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:06.941177 353 raft_consensus.cc:2258] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:06.943430 353 ts_tablet_manager.cc:1905] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:06.928691 552 consensus_queue.cc:237] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.944479 593 raft_consensus.cc:2781] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:06.946038 593 raft_consensus.cc:686] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:30:06.946795 593 consensus_queue.cc:237] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:06.952561 31938 catalog_manager.cc:6565] ReplaceTablet: tablet 3d592b1c4520467791ffd9d36e0d97fd deleted and replaced by tablet 23d2b217278c481e8464d00188c70048
I20260809 14:30:06.972195 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 2af8b48c8cde4f11a53eec939a1f8e3d
I20260809 14:30:06.973562 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3705cf965d214eb794984b9c16604326" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:06.975037 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8b48c8cde4f11a53eec939a1f8e3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:06.983901 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3705cf965d214eb794984b9c16604326" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:06.985219 32512 raft_consensus.cc:3037] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:06.999011 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_90fd7293a898493897859b855282cbbc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:07.004578 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_971c53b113a249d28e56fd3f27219c60" } new_table_name: "default.Table_4b848656d0ca4f2b81bd9e56d4540967" modify_external_catalogs: true
I20260809 14:30:07.006601 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 9922cad6d10945d5b013a168a3496676
W20260809 14:30:07.002816 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90fd7293a898493897859b855282cbbc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:07.010344 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 9922cad6d10945d5b013a168a3496676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:07.008343 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_d425a68ae37b4c39adf325216d4359ba" } new_table_name: "default.Table_569de184b14243abb58d1414941be5aa" modify_external_catalogs: true
I20260809 14:30:07.014349 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd with delete_type TABLET_DATA_DELETED (replaced by tablet 23d2b217278c481e8464d00188c70048) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:07.016350 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd with delete_type TABLET_DATA_DELETED (replaced by tablet 23d2b217278c481e8464d00188c70048) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:07.024775 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8b48c8cde4f11a53eec939a1f8e3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:07.027406 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8b48c8cde4f11a53eec939a1f8e3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:06.892671 32693 consensus_queue.cc:260] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:06.933749 32691 raft_consensus.cc:729] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:07.040243 32691 consensus_queue.cc:260] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:07.042392 32260 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 62947.8us
I20260809 14:30:07.044150 32296 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd with delete_type TABLET_DATA_DELETED (replaced by tablet 23d2b217278c481e8464d00188c70048) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:07.048975 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 9922cad6d10945d5b013a168a3496676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:07.045418 32693 ts_tablet_manager.cc:1428] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.383s user 0.008s sys 0.002s
I20260809 14:30:07.052109 32693 tablet_bootstrap.cc:492] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:07.063906 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_6b3c6d27f787472981f7207081957411"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:07.055976 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet 9922cad6d10945d5b013a168a3496676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:07.047899 32512 raft_consensus.cc:2454] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:07.060155 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 106) took 5036 ms. Trace:
I20260809 14:30:07.077883 426 rpcz_store.cc:276] 0809 14:30:02.023621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.023738 (+ 117us) service_pool.cc:227] Handling call
0809 14:30:06.265791 (+4242053us) ts_tablet_manager.cc:1145] Deleting tablet bfb1dd7b6b6243b4ba3eee9b1b3b1f1c
0809 14:30:06.265824 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:06.425669 (+159845us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:06.580863 (+155194us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.060086 (+479223us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4241435}
I20260809 14:30:07.079564 426 tablet_replica.cc:330] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:07.047180 353 ts_tablet_manager.cc:1918] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:07.081353 426 raft_consensus.cc:2229] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:07.081571 353 log.cc:1196] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/01cfc9e0319748c8b0e928fc64706933
I20260809 14:30:07.081876 426 raft_consensus.cc:2258] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.082386 353 ts_tablet_manager.cc:1939] T 01cfc9e0319748c8b0e928fc64706933 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:07.085163 613 raft_consensus.cc:482] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.085813 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 102) took 4181 ms. Trace:
I20260809 14:30:07.086187 353 rpcz_store.cc:276] 0809 14:30:02.904073 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.904197 (+ 124us) service_pool.cc:227] Handling call
0809 14:30:06.934749 (+4030552us) ts_tablet_manager.cc:1145] Deleting tablet 01cfc9e0319748c8b0e928fc64706933
0809 14:30:06.934807 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.002618 (+ 67811us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.046883 (+ 44265us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.085754 (+ 38871us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1330,"tablet-delete.queue_time_us":4029510}
I20260809 14:30:07.087424 353 tablet_replica.cc:330] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:07.087760 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 01cfc9e0319748c8b0e928fc64706933 (table default.Table_3b3fded95fa74c4086529b42834a0955 [id=f383923a235841a4a476b6cf31d33ef9]) successfully deleted
I20260809 14:30:07.088152 353 raft_consensus.cc:2229] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:07.087924 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_22a408c4e94c43a29e31198ad59a5e1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:07.089071 353 raft_consensus.cc:2258] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.086006 613 raft_consensus.cc:504] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.090921 353 ts_tablet_manager.cc:1905] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:07.100951 426 ts_tablet_manager.cc:1905] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:07.120136 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b3c6d27f787472981f7207081957411 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:07.121553 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22a408c4e94c43a29e31198ad59a5e1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:07.128010 32260 leader_election.cc:341] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:07.128134 353 ts_tablet_manager.cc:1918] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:07.094009 613 leader_election.cc:288] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:07.133853 353 log.cc:1196] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/658ab8e11b37419ba2176a9175bfd936
I20260809 14:30:07.134670 353 ts_tablet_manager.cc:1939] T 658ab8e11b37419ba2176a9175bfd936 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:07.136744 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 103) took 4230 ms. Trace:
I20260809 14:30:07.137077 353 rpcz_store.cc:276] 0809 14:30:02.905912 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.906041 (+ 129us) service_pool.cc:227] Handling call
0809 14:30:07.087274 (+4181233us) ts_tablet_manager.cc:1145] Deleting tablet 658ab8e11b37419ba2176a9175bfd936
0809 14:30:07.087315 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.120843 (+ 33528us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.127914 (+ 7071us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.136682 (+ 8768us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4180433}
I20260809 14:30:07.141867 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 104) took 4234 ms. Trace:
I20260809 14:30:07.142176 353 rpcz_store.cc:276] 0809 14:30:02.907367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.907491 (+ 124us) service_pool.cc:227] Handling call
0809 14:30:07.141498 (+4234007us) ts_tablet_manager.cc:1145] Deleting tablet 79bb4dd74afe4ace94eb1519950a83c0
0809 14:30:07.141540 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.141816 (+ 276us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4233303}
I20260809 14:30:07.143162 353 tablet_replica.cc:330] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
W20260809 14:30:07.143721 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 79bb4dd74afe4ace94eb1519950a83c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79bb4dd74afe4ace94eb1519950a83c0 already in progress: creating tablet
I20260809 14:30:07.143921 353 raft_consensus.cc:2229] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:07.144372 353 raft_consensus.cc:2258] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.146329 353 ts_tablet_manager.cc:1905] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:07.148635 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 3d592b1c4520467791ffd9d36e0d97fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d592b1c4520467791ffd9d36e0d97fd already in progress: creating tablet
W20260809 14:30:07.149550 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 2af8b48c8cde4f11a53eec939a1f8e3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2af8b48c8cde4f11a53eec939a1f8e3d already in progress: creating tablet
W20260809 14:30:07.150405 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 9922cad6d10945d5b013a168a3496676 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9922cad6d10945d5b013a168a3496676 already in progress: creating tablet
I20260809 14:30:07.163209 32260 leader_election.cc:302] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:07.175127 620 raft_consensus.cc:2781] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:07.177244 620 raft_consensus.cc:686] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:07.178141 620 consensus_queue.cc:237] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.181457 353 ts_tablet_manager.cc:1918] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:07.181624 32326 raft_consensus.cc:3037] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:07.181821 353 log.cc:1196] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/13f6c6a2ff894ec3a795f0c3f044ec17
I20260809 14:30:07.186211 353 ts_tablet_manager.cc:1939] T 13f6c6a2ff894ec3a795f0c3f044ec17 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:07.199496 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 98) took 4394 ms. Trace:
I20260809 14:30:07.199851 353 rpcz_store.cc:276] 0809 14:30:02.855647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.859010 (+ 3363us) service_pool.cc:227] Handling call
0809 14:30:07.143017 (+4284007us) ts_tablet_manager.cc:1145] Deleting tablet 13f6c6a2ff894ec3a795f0c3f044ec17
0809 14:30:07.143057 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.162357 (+ 19300us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.181227 (+ 18870us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.199418 (+ 18191us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11130,"tablet-delete.queue_time_us":4218310}
I20260809 14:30:07.201220 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 100) took 4299 ms. Trace:
I20260809 14:30:07.201557 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 13f6c6a2ff894ec3a795f0c3f044ec17 (table default.Table_5fdc5800b255423da5d44ad4ab4a1c2a [id=20d2073f354649b3b4bc4d583660c93a]) successfully deleted
I20260809 14:30:07.201545 353 rpcz_store.cc:276] 0809 14:30:02.901899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.924914 (+ 23015us) service_pool.cc:227] Handling call
0809 14:30:07.200964 (+4276050us) ts_tablet_manager.cc:1145] Deleting tablet a9f61933c2004a43af2c748ea445942c
0809 14:30:07.201008 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.201168 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4274877}
I20260809 14:30:07.202811 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 114) took 4085 ms. Trace:
W20260809 14:30:07.202898 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet a9f61933c2004a43af2c748ea445942c because the tablet was not found. No further retry: Not found: Tablet not found: a9f61933c2004a43af2c748ea445942c
I20260809 14:30:07.203146 353 rpcz_store.cc:276] 0809 14:30:03.117804 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.118155 (+ 351us) service_pool.cc:227] Handling call
0809 14:30:07.202453 (+4084298us) ts_tablet_manager.cc:1145] Deleting tablet 94c4cbdda4ed48a8b34728d231497d02
0809 14:30:07.202502 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.202757 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4036669}
I20260809 14:30:07.204190 353 tablet_replica.cc:330] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
W20260809 14:30:07.204572 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 94c4cbdda4ed48a8b34728d231497d02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94c4cbdda4ed48a8b34728d231497d02 already in progress: creating tablet
I20260809 14:30:07.205076 353 raft_consensus.cc:2229] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:07.208868 353 pending_rounds.cc:62] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 1 pending ops.
I20260809 14:30:07.209116 353 pending_rounds.cc:68] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626666210934784 op_type: NO_OP noop_request { }
I20260809 14:30:07.209504 353 raft_consensus.cc:2866] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:07.209841 353 raft_consensus.cc:2258] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:30:07.211772 32260 leader_election.cc:341] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Not found: Tablet not found: 422067edd08745bebdf59d4da704e665
I20260809 14:30:07.216518 32512 raft_consensus.cc:1261] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Refusing update from remote peer e1f94e0d792c46c9a0b6ab42820da6eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 14:30:07.227471 31948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5b7c498d7ea14448a8adbb89112bd384" } modify_external_catalogs: true
I20260809 14:30:07.227371 593 consensus_queue.cc:1035] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:07.228209 31948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5b7c498d7ea14448a8adbb89112bd384" } modify_external_catalogs: true
I20260809 14:30:07.233898 31946 catalog_manager.cc:5495] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65). New cstate: current_term: 1 leader_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:07.235572 31946 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
W20260809 14:30:07.252816 31938 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3d81046d929c48a3a6946f20d8fd7bca consensus state has no leader
W20260809 14:30:07.256937 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 88644048fa4a443fac09e951d78b3063 consensus state has no leader
I20260809 14:30:07.260077 32705 tablet_bootstrap.cc:492] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:07.260600 32705 ts_tablet_manager.cc:1397] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.451s user 0.020s sys 0.000s
W20260809 14:30:07.223233 628 consensus_queue.cc:1159] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260809 14:30:07.262549 353 ts_tablet_manager.cc:1905] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:07.263341 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_bbdad68122354be39969f430fdc7a0be" } modify_external_catalogs: true
I20260809 14:30:07.264398 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_bbdad68122354be39969f430fdc7a0be" } modify_external_catalogs: true
I20260809 14:30:07.266047 593 raft_consensus.cc:482] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.267292 32693 tablet_bootstrap.cc:654] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:07.266608 593 raft_consensus.cc:504] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.270246 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f36ab4503f24ca2a2a0d812c4c52468" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:07.270876 32514 raft_consensus.cc:2454] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:07.271790 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422067edd08745bebdf59d4da704e665" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
W20260809 14:30:07.271837 32481 consensus_meta.cc:287] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.511s user 0.011s sys 0.000s
I20260809 14:30:07.272715 32512 raft_consensus.cc:2379] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3eb0bc287b12421d89abf97ca66e4d9b in current term 1: Already voted for candidate e1f94e0d792c46c9a0b6ab42820da6eb in this term.
I20260809 14:30:07.275095 32095 leader_election.cc:302] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:07.275918 552 raft_consensus.cc:2781] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:07.276427 552 raft_consensus.cc:482] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:07.276970 552 raft_consensus.cc:3037] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:07.281185 32691 ts_tablet_manager.cc:1428] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.485s user 0.009s sys 0.000s
I20260809 14:30:07.282920 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f36ab4503f24ca2a2a0d812c4c52468" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:07.284396 32099 leader_election.cc:341] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:07.285187 593 leader_election.cc:288] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:07.288846 32691 tablet_bootstrap.cc:492] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:07.295892 32260 leader_election.cc:302] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:07.296789 353 ts_tablet_manager.cc:1918] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:07.297207 620 raft_consensus.cc:3037] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:07.297216 353 log.cc:1196] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/3705cf965d214eb794984b9c16604326
I20260809 14:30:07.298442 353 ts_tablet_manager.cc:1939] T 3705cf965d214eb794984b9c16604326 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:07.300673 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 113) took 4183 ms. Trace:
I20260809 14:30:07.300942 353 rpcz_store.cc:276] 0809 14:30:03.116926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.118362 (+ 1436us) service_pool.cc:227] Handling call
0809 14:30:07.204067 (+4085705us) ts_tablet_manager.cc:1145] Deleting tablet 3705cf965d214eb794984b9c16604326
0809 14:30:07.204102 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.271039 (+ 66937us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.296578 (+ 25539us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.300622 (+ 4044us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4036973}
I20260809 14:30:07.302075 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 115) took 3629 ms. Trace:
I20260809 14:30:07.303085 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 3705cf965d214eb794984b9c16604326 (table default.Table_f7fd4627a85c4f358f737fad5077415c [id=12165d071bb546219c54224908e47280]) successfully deleted
W20260809 14:30:07.316063 31931 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8b53f7d398d64701b4504db5c9c440b8 consensus state has no leader
I20260809 14:30:07.321158 353 rpcz_store.cc:276] 0809 14:30:03.672493 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.704099 (+ 31606us) service_pool.cc:227] Handling call
0809 14:30:07.301864 (+3597765us) ts_tablet_manager.cc:1145] Deleting tablet 71aad93fcc8b42e6b55db6b3ed6b4f91
0809 14:30:07.301899 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.302034 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3561078}
I20260809 14:30:07.322414 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 123) took 3175 ms. Trace:
I20260809 14:30:07.322716 353 rpcz_store.cc:276] 0809 14:30:04.146912 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.147093 (+ 181us) service_pool.cc:227] Handling call
0809 14:30:07.322063 (+3174970us) ts_tablet_manager.cc:1145] Deleting tablet 609b3e6049ec4b0582d78037b7416710
0809 14:30:07.322108 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.322362 (+ 254us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3174080}
I20260809 14:30:07.323650 353 tablet_replica.cc:330] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:07.327083 608 raft_consensus.cc:482] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.315231 552 raft_consensus.cc:504] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.327796 608 raft_consensus.cc:504] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.330643 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f36ab4503f24ca2a2a0d812c4c52468" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:07.331457 32512 raft_consensus.cc:3037] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:07.331068 608 leader_election.cc:288] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:07.328516 426 ts_tablet_manager.cc:1918] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:07.335330 426 log.cc:1196] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/658ab8e11b37419ba2176a9175bfd936
I20260809 14:30:07.324374 353 raft_consensus.cc:2229] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:07.336229 426 ts_tablet_manager.cc:1939] T 658ab8e11b37419ba2176a9175bfd936 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:07.289856 32326 raft_consensus.cc:1261] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Refusing update from remote peer e1f94e0d792c46c9a0b6ab42820da6eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 14:30:07.310093 32705 raft_consensus.cc:348] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.344899 32705 raft_consensus.cc:374] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:07.345239 32705 raft_consensus.cc:729] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:07.345211 608 raft_consensus.cc:482] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.345777 608 raft_consensus.cc:504] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.346086 32705 consensus_queue.cc:260] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:07.293326 32483 consensus_meta.cc:287] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.536s user 0.011s sys 0.001s
I20260809 14:30:07.351619 608 leader_election.cc:288] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:07.352766 31937 master_service.cc:946] ReplaceTablet: received request to replace tablet 29fdaef8a95e4aa0b23018d384442d16 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:07.357882 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ac3ef5573c40b894c9b2e743ca0582" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:07.360774 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "810c39c60b1942e2ab1e48624aeb0ee2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:07.364485 32439 leader_election.cc:341] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:07.365247 32439 leader_election.cc:341] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:07.372921 31930 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 221036us
I20260809 14:30:07.381814 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_60564937e4494c0ca93832cb7817a76c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:07.383682 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60564937e4494c0ca93832cb7817a76c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:07.387338 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 71aad93fcc8b42e6b55db6b3ed6b4f91 because the tablet was not found. No further retry: Not found: Tablet not found: 71aad93fcc8b42e6b55db6b3ed6b4f91
W20260809 14:30:07.388113 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 609b3e6049ec4b0582d78037b7416710 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 609b3e6049ec4b0582d78037b7416710 already in progress: creating tablet
I20260809 14:30:07.371048 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ac3ef5573c40b894c9b2e743ca0582" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:07.372381 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "810c39c60b1942e2ab1e48624aeb0ee2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:07.390530 32443 leader_election.cc:341] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:07.390942 32443 leader_election.cc:302] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:07.393215 32443 leader_election.cc:341] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:07.394158 32443 leader_election.cc:302] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:07.395583 353 raft_consensus.cc:2258] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.395627 617 raft_consensus.cc:2726] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:07.396329 579 raft_consensus.cc:482] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.397464 353 ts_tablet_manager.cc:1905] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:07.397115 579 raft_consensus.cc:504] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.398042 617 raft_consensus.cc:2726] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:07.400666 579 leader_election.cc:288] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:07.402644 593 consensus_queue.cc:1035] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:07.403801 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:07.407773 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:07.410351 32260 leader_election.cc:302] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:07.416344 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 107) took 5365 ms. Trace:
I20260809 14:30:07.416792 426 rpcz_store.cc:276] 0809 14:30:02.052278 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.052407 (+ 129us) service_pool.cc:227] Handling call
0809 14:30:07.079393 (+5026986us) ts_tablet_manager.cc:1145] Deleting tablet 658ab8e11b37419ba2176a9175bfd936
0809 14:30:07.079448 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.190177 (+110729us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.328252 (+138075us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.416274 (+ 88022us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":999,"tablet-delete.queue_time_us":5026334}
I20260809 14:30:07.421499 32693 tablet_bootstrap.cc:492] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:07.421929 32693 ts_tablet_manager.cc:1397] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.370s user 0.015s sys 0.000s
I20260809 14:30:07.425858 32693 raft_consensus.cc:348] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.426586 32693 raft_consensus.cc:374] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:07.426868 32693 raft_consensus.cc:729] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:07.427676 32693 consensus_queue.cc:260] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.429667 32693 ts_tablet_manager.cc:1428] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 14:30:07.431316 32693 tablet_bootstrap.cc:492] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:07.437536 610 mvcc.cc:205] Tried to move back new op lower bound from 7316626666028101632 to 7316626662396563456. Current Snapshot: MvccSnapshot[applied={T|T < 7316626666028101632}]
I20260809 14:30:07.466989 31946 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8df47c7eeba1459b8dceb9d0c3e49165" } new_table_name: "default.Table_1a59c65c45d844c6ad1424a0c36de6e7" modify_external_catalogs: true
I20260809 14:30:07.469208 31945 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b80dbb38192f49f797ea56d783e01f17" } new_table_name: "default.Table_3f10c680e2bc43569707615d0eb1dd66" modify_external_catalogs: true
I20260809 14:30:07.472815 621 mvcc.cc:205] Tried to move back new op lower bound from 7316626666028101632 to 7316626662396563456. Current Snapshot: MvccSnapshot[applied={T|T < 7316626666028101632}]
I20260809 14:30:07.475069 613 raft_consensus.cc:482] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.475852 613 raft_consensus.cc:504] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.479128 613 leader_election.cc:288] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:07.482162 353 ts_tablet_manager.cc:1918] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:07.485025 552 leader_election.cc:288] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:07.495517 32691 tablet_bootstrap.cc:654] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:07.497352 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:07.503770 593 raft_consensus.cc:482] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.508821 353 log.cc:1196] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/da7b2e6ca78b4fffb6f8daff1d874afe
I20260809 14:30:07.486824 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f36ab4503f24ca2a2a0d812c4c52468" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:07.509541 353 ts_tablet_manager.cc:1939] T da7b2e6ca78b4fffb6f8daff1d874afe P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:07.511350 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 122) took 3365 ms. Trace:
I20260809 14:30:07.511660 353 rpcz_store.cc:276] 0809 14:30:04.146347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.148256 (+ 1909us) service_pool.cc:227] Handling call
0809 14:30:07.323514 (+3175258us) ts_tablet_manager.cc:1145] Deleting tablet da7b2e6ca78b4fffb6f8daff1d874afe
0809 14:30:07.323552 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.394989 (+ 71437us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b540010eda0. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8398016471 00007f839801f40d 00007f839800d050 00007f8398070aba 00007f8398099375 00007f83984af26c 00007f83990bccfa 00007f83990ece10 00007f83990d3e0d 00007f83990d3db9 00007f83990d3d49
0809 14:30:07.407741 (+ 12752us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.481959 (+ 74218us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.511300 (+ 29341us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":131708800,"tablet-delete.queue_time_us":3174554}
I20260809 14:30:07.491719 32693 tablet_bootstrap.cc:654] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:07.513110 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet da7b2e6ca78b4fffb6f8daff1d874afe (table default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]) successfully deleted
W20260809 14:30:07.510576 32099 leader_election.cc:341] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:07.515336 593 raft_consensus.cc:504] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.520085 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:07.521250 626 tablet_replica.cc:330] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:07.520696 636 tablet.cc:1722] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 14:30:07.522320 579 raft_consensus.cc:2726] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:07.526082 593 leader_election.cc:288] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:07.531478 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 124) took 3373 ms. Trace:
I20260809 14:30:07.533308 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:07.533509 626 raft_consensus.cc:2229] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:07.542069 626 pending_rounds.cc:62] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:30:07.542460 626 pending_rounds.cc:68] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626665267740672 op_type: NO_OP noop_request { }
I20260809 14:30:07.542958 626 raft_consensus.cc:2866] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:07.543351 626 raft_consensus.cc:2258] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.546733 626 ts_tablet_manager.cc:1905] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:07.531889 353 rpcz_store.cc:276] 0809 14:30:04.158303 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.158498 (+ 195us) service_pool.cc:227] Handling call
0809 14:30:07.531079 (+3372581us) ts_tablet_manager.cc:1145] Deleting tablet 07ac3ef5573c40b894c9b2e743ca0582
0809 14:30:07.531141 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.531423 (+ 282us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3371595}
W20260809 14:30:07.550781 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 07ac3ef5573c40b894c9b2e743ca0582 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07ac3ef5573c40b894c9b2e743ca0582 already in progress: creating tablet
I20260809 14:30:07.552397 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 126) took 3250 ms. Trace:
I20260809 14:30:07.552822 353 rpcz_store.cc:276] 0809 14:30:04.301729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.302324 (+ 595us) service_pool.cc:227] Handling call
0809 14:30:07.551984 (+3249660us) ts_tablet_manager.cc:1145] Deleting tablet 969849d30c204937a7508e8d54dae427
0809 14:30:07.552046 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.552324 (+ 278us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3248748}
W20260809 14:30:07.554500 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 969849d30c204937a7508e8d54dae427 because the tablet was not found. No further retry: Not found: Tablet not found: 969849d30c204937a7508e8d54dae427
I20260809 14:30:07.557293 32705 ts_tablet_manager.cc:1428] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.296s user 0.006s sys 0.001s
I20260809 14:30:07.557297 353 tablet_replica.cc:330] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:07.565763 353 raft_consensus.cc:2229] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:07.569865 32705 tablet_bootstrap.cc:492] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:07.572023 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:07.572590 353 pending_rounds.cc:62] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 1 pending ops.
I20260809 14:30:07.572789 353 pending_rounds.cc:68] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626656678281216 op_type: NO_OP noop_request { }
I20260809 14:30:07.573153 353 raft_consensus.cc:2866] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:07.573448 353 raft_consensus.cc:2258] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.575131 353 ts_tablet_manager.cc:1905] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:07.567143 641 tablet.cc:1722] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 14:30:07.606765 31948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet d4980140ba2a4ecc87001e2a43c8e813
I20260809 14:30:07.607543 32693 tablet_bootstrap.cc:492] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:07.607947 32693 ts_tablet_manager.cc:1397] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.177s user 0.007s sys 0.012s
I20260809 14:30:07.610183 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet d4980140ba2a4ecc87001e2a43c8e813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:07.612201 31942 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet d1ef90e75341457da6d7fc4a86e80451
I20260809 14:30:07.619532 32693 raft_consensus.cc:348] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.621773 31942 master_service.cc:946] ReplaceTablet: received request to replace tablet 6fa25dbd78f64e5db9e31d433e8f4736 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:07.620330 32693 raft_consensus.cc:374] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:07.624269 32693 raft_consensus.cc:729] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:07.629041 32693 consensus_queue.cc:260] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.631978 589 raft_consensus.cc:482] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.632750 589 raft_consensus.cc:504] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.644002 589 leader_election.cc:288] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:07.497623 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb799e344ae544f9b4df659462bddba2" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:07.646533 32514 raft_consensus.cc:2379] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3eb0bc287b12421d89abf97ca66e4d9b in current term 1: Already voted for candidate e1f94e0d792c46c9a0b6ab42820da6eb in this term.
I20260809 14:30:07.650022 32693 ts_tablet_manager.cc:1428] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.042s user 0.002s sys 0.008s
I20260809 14:30:07.650995 32693 tablet_bootstrap.cc:492] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:07.653483 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_bea4b6049d234a58a4bc8222fd20f462"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:07.657348 620 raft_consensus.cc:2726] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:07.664615 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b831dc978eb249f7b28215db2cc26ef0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:07.666532 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_250ec22f736f4ef885234ffff62f2054"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:07.501608 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 108) took 5151 ms. Trace:
I20260809 14:30:07.526425 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:07.669176 32515 raft_consensus.cc:2379] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:07.670398 32095 leader_election.cc:302] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:07.671679 426 rpcz_store.cc:276] 0809 14:30:02.350766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.430076 (+ 79310us) service_pool.cc:227] Handling call
0809 14:30:07.501153 (+5071077us) ts_tablet_manager.cc:1145] Deleting tablet 79bb4dd74afe4ace94eb1519950a83c0
0809 14:30:07.501196 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.501552 (+ 356us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5065628}
I20260809 14:30:07.676235 32691 tablet_bootstrap.cc:492] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:07.676406 426 tablet_replica.cc:330] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:07.676896 32691 ts_tablet_manager.cc:1397] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.391s user 0.006s sys 0.008s
I20260809 14:30:07.677726 552 raft_consensus.cc:3037] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:07.680385 32691 raft_consensus.cc:348] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.681474 32691 raft_consensus.cc:374] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:07.681931 32691 raft_consensus.cc:729] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:07.682850 32691 consensus_queue.cc:260] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.683907 32693 tablet_bootstrap.cc:654] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:07.536489 637 tablet.cc:1722] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 14:30:07.686858 353 ts_tablet_manager.cc:1918] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:07.687176 353 log.cc:1196] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/e378d7763c5046249011a6d6e0617c97
I20260809 14:30:07.687865 353 ts_tablet_manager.cc:1939] T e378d7763c5046249011a6d6e0617c97 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:07.567750 32512 raft_consensus.cc:2454] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
W20260809 14:30:07.690718 31896 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad240 after lost signal to thread 31821
W20260809 14:30:07.691742 31896 debug-util.cc:398] Leaking SignalData structure 0x7b08000ae980 after lost signal to thread 31899
I20260809 14:30:07.691861 32095 leader_election.cc:302] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:30:07.692766 611 raft_consensus.cc:2781] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:07.693285 611 raft_consensus.cc:686] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:30:07.694106 611 consensus_queue.cc:237] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
W20260809 14:30:07.696571 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b831dc978eb249f7b28215db2cc26ef0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:07.696233 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_250ec22f736f4ef885234ffff62f2054 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:07.704205 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bea4b6049d234a58a4bc8222fd20f462 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:07.710359 32691 ts_tablet_manager.cc:1428] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.033s user 0.005s sys 0.001s
I20260809 14:30:07.689805 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 127) took 3232 ms. Trace:
I20260809 14:30:07.710815 353 rpcz_store.cc:276] 0809 14:30:04.457648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.460725 (+ 3077us) service_pool.cc:227] Handling call
0809 14:30:07.557098 (+3096373us) ts_tablet_manager.cc:1145] Deleting tablet e378d7763c5046249011a6d6e0617c97
0809 14:30:07.557151 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.626953 (+ 69802us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.686648 (+ 59695us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.689739 (+ 3091us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3095219}
I20260809 14:30:07.718808 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f36ab4503f24ca2a2a0d812c4c52468" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:07.721169 426 raft_consensus.cc:2229] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:07.721725 426 raft_consensus.cc:2258] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.723523 426 ts_tablet_manager.cc:1905] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:07.724241 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet e378d7763c5046249011a6d6e0617c97 (table default.Table_3ecd77174e9948a59988f0b186dc8793 [id=82601d705044402cbf2820f18668da15]) successfully deleted
I20260809 14:30:07.725467 353 tablet_replica.cc:330] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:07.726172 353 raft_consensus.cc:2229] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:07.726979 353 raft_consensus.cc:2258] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.728706 353 ts_tablet_manager.cc:1905] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:07.761677 32693 tablet_bootstrap.cc:492] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:07.762102 32693 ts_tablet_manager.cc:1397] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.111s user 0.010s sys 0.009s
I20260809 14:30:07.765764 32260 leader_election.cc:302] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:07.766712 620 raft_consensus.cc:3037] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:07.780392 620 raft_consensus.cc:2726] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:07.724265 32691 tablet_bootstrap.cc:492] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:07.765637 32693 raft_consensus.cc:348] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.790030 589 consensus_queue.cc:1035] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:07.800671 32693 raft_consensus.cc:374] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:07.800956 32691 tablet_bootstrap.cc:654] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:07.801080 32693 raft_consensus.cc:729] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:30:07.802343 31945 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67afcb47ffc947c5b1d98c96de333116 consensus state has no leader
I20260809 14:30:07.801707 32693 consensus_queue.cc:260] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:07.764544 32512 raft_consensus.cc:2379] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3eb0bc287b12421d89abf97ca66e4d9b in current term 1: Already voted for candidate e1f94e0d792c46c9a0b6ab42820da6eb in this term.
I20260809 14:30:07.785462 32483 tablet_service.cc:1514] Processing DeleteTablet for tablet d4980140ba2a4ecc87001e2a43c8e813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:07.808041 32693 ts_tablet_manager.cc:1428] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.046s user 0.008s sys 0.000s
W20260809 14:30:07.808969 31946 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2c2fbb10b4ae46eb9096fe9f665fa6f9 has no consensus state
I20260809 14:30:07.811477 353 ts_tablet_manager.cc:1918] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:07.811895 353 log.cc:1196] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/849d798165144531810a03dc2b9b5f1e
I20260809 14:30:07.813107 32693 tablet_bootstrap.cc:492] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:07.818535 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9d422f6e103b4228825e2f4398b4e1e7" } modify_external_catalogs: true
I20260809 14:30:07.819097 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9d422f6e103b4228825e2f4398b4e1e7" } modify_external_catalogs: true
I20260809 14:30:07.821240 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f4fdd6821b4749b4869ebd018c1e7ed7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:07.829540 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4fdd6821b4749b4869ebd018c1e7ed7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:07.832624 31930 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 43862.4us
I20260809 14:30:07.832937 353 ts_tablet_manager.cc:1939] T 849d798165144531810a03dc2b9b5f1e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
W20260809 14:30:07.841722 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 149961us
I20260809 14:30:07.844982 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 128) took 3218 ms. Trace:
I20260809 14:30:07.845396 353 rpcz_store.cc:276] 0809 14:30:04.626872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.627217 (+ 345us) service_pool.cc:227] Handling call
0809 14:30:07.725313 (+3098096us) ts_tablet_manager.cc:1145] Deleting tablet 849d798165144531810a03dc2b9b5f1e
0809 14:30:07.725354 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.766587 (+ 41233us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.811225 (+ 44638us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:07.844867 (+ 33642us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3097325}
I20260809 14:30:07.847046 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]) successfully deleted
I20260809 14:30:07.859036 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 129) took 3135 ms. Trace:
I20260809 14:30:07.859455 353 rpcz_store.cc:276] 0809 14:30:04.719877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.720037 (+ 160us) service_pool.cc:227] Handling call
0809 14:30:07.855017 (+3134980us) ts_tablet_manager.cc:1145] Deleting tablet faf222c49c934d52a2642f20e8575702
0809 14:30:07.855072 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.855366 (+ 294us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3134323}
W20260809 14:30:07.859946 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 79bb4dd74afe4ace94eb1519950a83c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79bb4dd74afe4ace94eb1519950a83c0 already in progress: creating tablet
W20260809 14:30:07.860759 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet faf222c49c934d52a2642f20e8575702 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet faf222c49c934d52a2642f20e8575702 already in progress: creating tablet
I20260809 14:30:07.861621 32693 tablet_bootstrap.cc:654] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:07.868676 353 tablet_replica.cc:330] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:07.869607 353 raft_consensus.cc:2229] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:07.873096 353 raft_consensus.cc:2258] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:07.877022 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet d4980140ba2a4ecc87001e2a43c8e813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:07.877605 353 ts_tablet_manager.cc:1905] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:07.885305 593 consensus_queue.cc:1035] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 14:30:07.875422 608 raft_consensus.cc:482] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:07.889631 608 raft_consensus.cc:504] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.906764 608 leader_election.cc:288] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:07.906100 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:07.912886 32443 leader_election.cc:341] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:07.918061 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f36ab4503f24ca2a2a0d812c4c52468" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:07.923448 32705 tablet_bootstrap.cc:654] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:07.928292 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:07.933444 32439 leader_election.cc:341] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:07.933933 32439 leader_election.cc:302] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:07.935231 608 raft_consensus.cc:2726] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:07.939275 32260 leader_election.cc:302] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:07.940186 589 raft_consensus.cc:3037] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:07.954216 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_52d55a3e0baf4604a7f9330e03b5b3eb" } modify_external_catalogs: true
I20260809 14:30:07.954877 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_52d55a3e0baf4604a7f9330e03b5b3eb" } modify_external_catalogs: true
I20260809 14:30:07.955147 589 raft_consensus.cc:2726] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:07.939529 32693 tablet_bootstrap.cc:492] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:07.956655 32693 ts_tablet_manager.cc:1397] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.144s user 0.010s sys 0.009s
W20260809 14:30:07.961133 31930 consensus_peers.cc:487] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 -> Peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Couldn't send request to peer 9d10512ac8b3406098fad1f7da012ab3. Status: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260809 14:30:07.964669 32691 tablet_bootstrap.cc:492] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:07.965153 32691 ts_tablet_manager.cc:1397] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.241s user 0.018s sys 0.014s
I20260809 14:30:07.968156 32691 raft_consensus.cc:348] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.969166 32691 raft_consensus.cc:374] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:07.969467 32691 raft_consensus.cc:729] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:07.970182 32691 consensus_queue.cc:260] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.978974 32693 raft_consensus.cc:348] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.979770 32693 raft_consensus.cc:374] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:07.980072 32693 raft_consensus.cc:729] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:07.988912 32693 consensus_queue.cc:260] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:07.991200 32693 ts_tablet_manager.cc:1428] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260809 14:30:07.992091 32693 tablet_bootstrap.cc:492] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:07.992242 32691 ts_tablet_manager.cc:1428] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.027s user 0.012s sys 0.001s
I20260809 14:30:07.994609 32691 tablet_bootstrap.cc:492] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:08.007717 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8b32f662e5154cf78a133dd5249f5f0f" } new_table_name: "default.Table_ee3df66954244dd79fb6af9208969ed8" modify_external_catalogs: true
I20260809 14:30:08.019593 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_2aaadcb724904194bbbb379d4a2da394" } new_table_name: "default.Table_45d346a083d04fc6b8a9d8631c65bff1" modify_external_catalogs: true
I20260809 14:30:08.020876 32691 tablet_bootstrap.cc:654] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:08.047757 552 raft_consensus.cc:2726] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:08.052593 32691 tablet_bootstrap.cc:492] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:08.053002 32691 ts_tablet_manager.cc:1397] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.006s
I20260809 14:30:08.055462 32691 raft_consensus.cc:348] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.056175 32691 raft_consensus.cc:374] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:08.056468 32691 raft_consensus.cc:729] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:08.057380 32693 tablet_bootstrap.cc:654] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:08.065464 353 ts_tablet_manager.cc:1918] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:08.065807 353 log.cc:1196] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/84c8eb8ad2dd406085779ed6a86eb70e
I20260809 14:30:08.066497 353 ts_tablet_manager.cc:1939] T 84c8eb8ad2dd406085779ed6a86eb70e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:08.067451 32691 consensus_queue.cc:260] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.081941 32691 ts_tablet_manager.cc:1428] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20260809 14:30:08.086354 32691 tablet_bootstrap.cc:492] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:08.105546 32705 tablet_bootstrap.cc:492] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:08.106122 32705 ts_tablet_manager.cc:1397] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.536s user 0.009s sys 0.007s
I20260809 14:30:08.106950 32691 tablet_bootstrap.cc:654] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:08.058888 589 raft_consensus.cc:482] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1f94e0d792c46c9a0b6ab42820da6eb)
I20260809 14:30:08.108191 589 raft_consensus.cc:504] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:08.125824 620 consensus_queue.cc:1035] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 14:30:08.144438 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 130) took 2811 ms. Trace:
I20260809 14:30:08.145212 353 rpcz_store.cc:276] 0809 14:30:05.332782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.333077 (+ 295us) service_pool.cc:227] Handling call
0809 14:30:07.868445 (+2535368us) ts_tablet_manager.cc:1145] Deleting tablet 84c8eb8ad2dd406085779ed6a86eb70e
0809 14:30:07.868526 (+ 81us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:07.962727 (+ 94201us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.065249 (+102522us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.144346 (+ 79097us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2534576}
I20260809 14:30:08.146909 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 84c8eb8ad2dd406085779ed6a86eb70e (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
W20260809 14:30:08.148530 32260 leader_election.cc:341] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Not found: Tablet not found: 27e2d8ea00d247f69c14e04728519f8b
I20260809 14:30:08.149910 589 leader_election.cc:288] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:08.150833 426 ts_tablet_manager.cc:1918] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:08.151235 426 log.cc:1196] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/3705cf965d214eb794984b9c16604326
I20260809 14:30:08.151932 426 ts_tablet_manager.cc:1939] T 3705cf965d214eb794984b9c16604326 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:08.149685 32705 raft_consensus.cc:348] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:08.168622 32705 raft_consensus.cc:374] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:08.168910 32705 raft_consensus.cc:729] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:08.173408 32705 consensus_queue.cc:260] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:08.182137 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9c8d59829d3a4c3fa74ab227cb412935"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:08.183625 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c8d59829d3a4c3fa74ab227cb412935 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:08.188585 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b393017627934fa1b740d8ba8ac62d51 consensus state has no leader
I20260809 14:30:08.190253 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 133) took 2673 ms. Trace:
W20260809 14:30:08.190372 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23d2b217278c481e8464d00188c70048 has no consensus state
I20260809 14:30:08.190541 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 60acf8a11d6143948bfe0040f74f5ef8 (DEFAULT_TABLE table=default.Table_9e998648ceee4a19835ede64b0ef1038 [id=217a91e1a58b443d93738772edc80b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.190615 353 rpcz_store.cc:276] 0809 14:30:05.516809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.516964 (+ 155us) service_pool.cc:227] Handling call
0809 14:30:08.148083 (+2631119us) ts_tablet_manager.cc:1145] Deleting tablet 4cdfc56edcd34696a7b806b0ee34d594
0809 14:30:08.148191 (+ 108us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.190191 (+ 42000us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2630356}
I20260809 14:30:08.191857 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 135) took 2623 ms. Trace:
I20260809 14:30:08.191962 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60acf8a11d6143948bfe0040f74f5ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.191515 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_fb1e9afadc124d26b601aeaaaf0d14b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:08.193998 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 60acf8a11d6143948bfe0040f74f5ef8 (DEFAULT_TABLE table=default.Table_9e998648ceee4a19835ede64b0ef1038 [id=217a91e1a58b443d93738772edc80b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.195772 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60acf8a11d6143948bfe0040f74f5ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.196872 31939 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet e28a765e1854486595683c0ac00da24c (table default.Table_5fdc5800b255423da5d44ad4ab4a1c2a [id=20d2073f354649b3b4bc4d583660c93a]). Expected version 3 got 0
I20260809 14:30:08.197341 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 2c2fbb10b4ae46eb9096fe9f665fa6f9 (DEFAULT_TABLE table=default.Table_1a59c65c45d844c6ad1424a0c36de6e7 [id=57abcf2570f14f039173f115a89c2d4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.198372 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c2fbb10b4ae46eb9096fe9f665fa6f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.199772 32693 tablet_bootstrap.cc:492] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:08.200160 32693 ts_tablet_manager.cc:1397] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.208s user 0.004s sys 0.008s
I20260809 14:30:08.200150 32481 tablet_service.cc:1467] Processing CreateTablet for tablet 2c2fbb10b4ae46eb9096fe9f665fa6f9 (DEFAULT_TABLE table=default.Table_1a59c65c45d844c6ad1424a0c36de6e7 [id=57abcf2570f14f039173f115a89c2d4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.202804 31938 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet e28a765e1854486595683c0ac00da24c (table default.Table_5fdc5800b255423da5d44ad4ab4a1c2a [id=20d2073f354649b3b4bc4d583660c93a]). Expected version 3 got 0
I20260809 14:30:08.204778 32693 raft_consensus.cc:348] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.205770 32693 raft_consensus.cc:374] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:08.206044 32693 raft_consensus.cc:729] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:08.208387 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 868a8000400147c89e9d919c179fd3a0 (DEFAULT_TABLE table=default.Table_a737e7da4d6445b7b9e29bc4a9fb13aa [id=9effad5f48cc48e5a24bf2cd91f1d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.208742 32693 consensus_queue.cc:260] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.192209 353 rpcz_store.cc:276] 0809 14:30:05.568995 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.569159 (+ 164us) service_pool.cc:227] Handling call
0809 14:30:08.191523 (+2622364us) ts_tablet_manager.cc:1145] Deleting tablet 56a7a1c8e38748cd9a3c4ccef2aea13c
0809 14:30:08.191563 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.191798 (+ 235us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2621299}
I20260809 14:30:08.210409 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868a8000400147c89e9d919c179fd3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.212086 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 5be5cc6d7cb042d1b577cb7b24a95827 (DEFAULT_TABLE table=default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.213315 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be5cc6d7cb042d1b577cb7b24a95827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.216848 353 tablet_replica.cc:330] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:08.217531 353 raft_consensus.cc:2229] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:08.218390 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 868a8000400147c89e9d919c179fd3a0 (DEFAULT_TABLE table=default.Table_a737e7da4d6445b7b9e29bc4a9fb13aa [id=9effad5f48cc48e5a24bf2cd91f1d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.217290 552 consensus_queue.cc:1035] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260809 14:30:08.224994 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 4cdfc56edcd34696a7b806b0ee34d594 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cdfc56edcd34696a7b806b0ee34d594 already in progress: creating tablet
W20260809 14:30:08.225741 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56a7a1c8e38748cd9a3c4ccef2aea13c already in progress: creating tablet
I20260809 14:30:08.227965 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 60acf8a11d6143948bfe0040f74f5ef8 (DEFAULT_TABLE table=default.Table_9e998648ceee4a19835ede64b0ef1038 [id=217a91e1a58b443d93738772edc80b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.227617 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 5be5cc6d7cb042d1b577cb7b24a95827 (DEFAULT_TABLE table=default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.229215 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60acf8a11d6143948bfe0040f74f5ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.228365 32300 tablet_service.cc:1467] Processing CreateTablet for tablet a39a169c5f7342f99bdfa7e7d2c01d4d (DEFAULT_TABLE table=default.Table_06a4ae5a60b44618a6f88f5ad5b71ac9 [id=4ecde42868384f93a2abe6720fdc3966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.230362 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a39a169c5f7342f99bdfa7e7d2c01d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.230189 32139 tablet_service.cc:1467] Processing CreateTablet for tablet a39a169c5f7342f99bdfa7e7d2c01d4d (DEFAULT_TABLE table=default.Table_06a4ae5a60b44618a6f88f5ad5b71ac9 [id=4ecde42868384f93a2abe6720fdc3966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.231105 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a39a169c5f7342f99bdfa7e7d2c01d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.233547 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 2c2fbb10b4ae46eb9096fe9f665fa6f9 (DEFAULT_TABLE table=default.Table_1a59c65c45d844c6ad1424a0c36de6e7 [id=57abcf2570f14f039173f115a89c2d4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.234660 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c2fbb10b4ae46eb9096fe9f665fa6f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.236346 626 ts_tablet_manager.cc:1918] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:08.237195 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 868a8000400147c89e9d919c179fd3a0 (DEFAULT_TABLE table=default.Table_a737e7da4d6445b7b9e29bc4a9fb13aa [id=9effad5f48cc48e5a24bf2cd91f1d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.238188 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868a8000400147c89e9d919c179fd3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.239566 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be5cc6d7cb042d1b577cb7b24a95827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.242934 32482 tablet_service.cc:1467] Processing CreateTablet for tablet 5be5cc6d7cb042d1b577cb7b24a95827 (DEFAULT_TABLE table=default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:08.257613 32260 leader_election.cc:341] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 27e2d8ea00d247f69c14e04728519f8b
I20260809 14:30:08.258364 32260 leader_election.cc:302] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:08.261385 32705 ts_tablet_manager.cc:1428] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.155s user 0.008s sys 0.001s
I20260809 14:30:08.262349 32705 tablet_bootstrap.cc:492] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:08.236922 626 log.cc:1196] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/56a7a1c8e38748cd9a3c4ccef2aea13c
I20260809 14:30:08.266964 353 raft_consensus.cc:2258] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:08.267481 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868a8000400147c89e9d919c179fd3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.268795 353 ts_tablet_manager.cc:1905] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:08.271721 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be5cc6d7cb042d1b577cb7b24a95827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.274153 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 117) took 5516 ms. Trace:
I20260809 14:30:08.274994 32693 ts_tablet_manager.cc:1428] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.075s user 0.009s sys 0.000s
I20260809 14:30:08.274745 426 rpcz_store.cc:276] 0809 14:30:02.757615 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:02.758044 (+ 429us) service_pool.cc:227] Handling call
0809 14:30:07.675818 (+4917774us) ts_tablet_manager.cc:1145] Deleting tablet 3705cf965d214eb794984b9c16604326
0809 14:30:07.675972 (+ 154us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.111606 (+435634us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.150560 (+ 38954us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.274090 (+123530us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12735,"tablet-delete.queue_time_us":4835084}
I20260809 14:30:08.275581 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c2fbb10b4ae46eb9096fe9f665fa6f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.276850 626 ts_tablet_manager.cc:1939] T 56a7a1c8e38748cd9a3c4ccef2aea13c P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:08.294307 32693 tablet_bootstrap.cc:492] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:08.302016 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 118) took 5188 ms. Trace:
I20260809 14:30:08.303527 626 tablet_replica.cc:330] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:08.302446 426 rpcz_store.cc:276] 0809 14:30:03.113299 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.141934 (+ 28635us) service_pool.cc:227] Handling call
0809 14:30:08.300412 (+5158478us) ts_tablet_manager.cc:1145] Deleting tablet 94c4cbdda4ed48a8b34728d231497d02
0809 14:30:08.300453 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.301931 (+ 1478us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5139663}
I20260809 14:30:08.306667 626 raft_consensus.cc:2229] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:08.311754 426 tablet_replica.cc:330] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:08.314518 32693 tablet_bootstrap.cc:654] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:08.317129 426 raft_consensus.cc:2229] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:08.317601 426 raft_consensus.cc:2258] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:08.318994 626 pending_rounds.cc:62] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:30:08.325039 626 pending_rounds.cc:68] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626668937523200 op_type: NO_OP noop_request { }
I20260809 14:30:08.326246 353 ts_tablet_manager.cc:1918] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:08.326710 626 raft_consensus.cc:2866] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:08.327340 353 log.cc:1196] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/27e2d8ea00d247f69c14e04728519f8b
I20260809 14:30:08.328415 426 ts_tablet_manager.cc:1905] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:08.328981 626 raft_consensus.cc:2258] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:08.331291 353 ts_tablet_manager.cc:1939] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:08.332804 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 141) took 2193 ms. Trace:
I20260809 14:30:08.335775 626 ts_tablet_manager.cc:1905] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:08.336180 353 rpcz_store.cc:276] 0809 14:30:06.138973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:06.139596 (+ 623us) service_pool.cc:227] Handling call
0809 14:30:08.216715 (+2077119us) ts_tablet_manager.cc:1145] Deleting tablet 27e2d8ea00d247f69c14e04728519f8b
0809 14:30:08.216754 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.292714 (+ 75960us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.326012 (+ 33298us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.332732 (+ 6720us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":47650,"tablet-delete.queue_time_us":2076345}
I20260809 14:30:08.345999 353 tablet_replica.cc:330] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:08.348593 32693 tablet_bootstrap.cc:492] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:08.349190 32693 ts_tablet_manager.cc:1397] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.055s user 0.012s sys 0.005s
I20260809 14:30:08.357375 353 raft_consensus.cc:2229] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:08.357995 353 raft_consensus.cc:2258] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:08.359611 353 ts_tablet_manager.cc:1905] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:08.360986 32693 raft_consensus.cc:348] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.369486 32693 raft_consensus.cc:374] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:08.372562 32693 raft_consensus.cc:729] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:08.374037 32693 consensus_queue.cc:260] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.379005 32693 ts_tablet_manager.cc:1428] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20260809 14:30:08.384192 32693 tablet_bootstrap.cc:492] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
W20260809 14:30:08.210157 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb1e9afadc124d26b601aeaaaf0d14b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:08.406209 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b393017627934fa1b740d8ba8ac62d51 (table default.Table_ee3df66954244dd79fb6af9208969ed8 [id=7043672a284145c7a5c9a5d80c322ef8], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b393017627934fa1b740d8ba8ac62d51 consensus state has no leader
I20260809 14:30:08.406701 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 3705cf965d214eb794984b9c16604326 (table default.Table_f7fd4627a85c4f358f737fad5077415c [id=12165d071bb546219c54224908e47280]) successfully deleted
I20260809 14:30:08.407377 32484 tablet_service.cc:1467] Processing CreateTablet for tablet a39a169c5f7342f99bdfa7e7d2c01d4d (DEFAULT_TABLE table=default.Table_06a4ae5a60b44618a6f88f5ad5b71ac9 [id=4ecde42868384f93a2abe6720fdc3966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.408686 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a39a169c5f7342f99bdfa7e7d2c01d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.409253 32139 tablet_service.cc:1467] Processing CreateTablet for tablet eba8c216630d482db5bf235b4c8f5b15 (DEFAULT_TABLE table=default.Table_77a290e4278b410cbfbe30feb015ef83 [id=f604d61a6a204aa7ba0ae3124bd3dc9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.409698 32138 tablet_service.cc:1467] Processing CreateTablet for tablet a0b196e429514cbc803e9a99a986d4d2 (DEFAULT_TABLE table=default.Table_b9541d0208bb4c938aff912a0df84a22 [id=58e17b048a594020a069730291d81345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.410382 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eba8c216630d482db5bf235b4c8f5b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.410269 32486 tablet_service.cc:1467] Processing CreateTablet for tablet a0b196e429514cbc803e9a99a986d4d2 (DEFAULT_TABLE table=default.Table_b9541d0208bb4c938aff912a0df84a22 [id=58e17b048a594020a069730291d81345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.411823 32491 tablet_service.cc:1467] Processing CreateTablet for tablet eba8c216630d482db5bf235b4c8f5b15 (DEFAULT_TABLE table=default.Table_77a290e4278b410cbfbe30feb015ef83 [id=f604d61a6a204aa7ba0ae3124bd3dc9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.412838 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 03f6a524d4ef48c4a31e0bfec2f557bb (DEFAULT_TABLE table=default.Table_2c3e9ee14404475bba64491f9d132524 [id=07a8421008714d8eb96d3d29c06c8fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.413874 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03f6a524d4ef48c4a31e0bfec2f557bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.419306 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b196e429514cbc803e9a99a986d4d2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:08.430864 32260 raft_consensus.cc:2670] T 27e2d8ea00d247f69c14e04728519f8b P 3eb0bc287b12421d89abf97ca66e4d9b: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 14:30:08.440021 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 182382us
I20260809 14:30:08.441540 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_09716b0be3174d30bc3adf0383d40efd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:08.443104 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09716b0be3174d30bc3adf0383d40efd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:08.446035 31943 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
I20260809 14:30:08.447228 32307 tablet_service.cc:1467] Processing CreateTablet for tablet eba8c216630d482db5bf235b4c8f5b15 (DEFAULT_TABLE table=default.Table_77a290e4278b410cbfbe30feb015ef83 [id=f604d61a6a204aa7ba0ae3124bd3dc9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.449153 32303 tablet_service.cc:1467] Processing CreateTablet for tablet 200c48c11cb042169b307bc1902be82e (DEFAULT_TABLE table=default.Table_65d17735090446738a487e765aef028a [id=6d697fa0067f4a3c960023c24567477c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.449798 32298 tablet_service.cc:1467] Processing CreateTablet for tablet 3d7cbb2bb26c4bd7a089648007d7f041 (DEFAULT_TABLE table=default.Table_e8d440cfb3be48698670be598799bce1 [id=f0ca818a03d640338e363e544463efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.447501 32295 tablet_service.cc:1467] Processing CreateTablet for tablet b5396854feb14153a115f1425f8f4224 (DEFAULT_TABLE table=default.Table_7930652dac8a4ec885cd18e733b1e84e [id=1dd4e785ac184325be3876d8148799f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.451787 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eba8c216630d482db5bf235b4c8f5b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.456310 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 27e2d8ea00d247f69c14e04728519f8b (table default.Table_acb53b3bc5b24b989ccf44b01709c500 [id=4c072795da8241a18c26f3a01af4dda6]) successfully deleted
W20260809 14:30:08.456890 31939 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 48) took 875 ms (client timeout 1000 ms). Trace:
W20260809 14:30:08.457182 31939 rpcz_store.cc:269] 0809 14:30:07.581184 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.613970 (+ 32786us) service_pool.cc:227] Handling call
0809 14:30:08.456836 (+842866us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":4}
I20260809 14:30:08.458393 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b342ca5d31c14a89baa7ed0ea2bf8da8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:08.458788 32300 tablet_service.cc:1467] Processing CreateTablet for tablet 03f6a524d4ef48c4a31e0bfec2f557bb (DEFAULT_TABLE table=default.Table_2c3e9ee14404475bba64491f9d132524 [id=07a8421008714d8eb96d3d29c06c8fc5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:08.459774 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b342ca5d31c14a89baa7ed0ea2bf8da8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:08.460186 32303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 200c48c11cb042169b307bc1902be82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.461172 32304 tablet_service.cc:1514] Processing DeleteTablet for tablet da7b2e6ca78b4fffb6f8daff1d874afe with delete_type TABLET_DATA_DELETED (replaced by tablet 37a3a71874fd44bf865b11c05e84ca84) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:08.462056 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e0df5c5c4f9a474d9ba2d8ab9f79c26d" } new_table_name: "default.Table_b86f8f5a1acc4bacb6b93b4acd413b0f" modify_external_catalogs: true
W20260809 14:30:08.464625 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 94c4cbdda4ed48a8b34728d231497d02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94c4cbdda4ed48a8b34728d231497d02 already in progress: creating tablet
I20260809 14:30:08.465009 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c8eb8ad2dd406085779ed6a86eb70e with delete_type TABLET_DATA_DELETED (replaced by tablet c8a95e406071421cac1704a42a5f78ff) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:08.465131 31930 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 57871.9us
I20260809 14:30:08.466485 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 56a7a1c8e38748cd9a3c4ccef2aea13c (table default.Table_f3f904c0b94440ddb1f6f7f733689d8c [id=ae6b2507dcdd4367b1faa278e84fbe3a]) successfully deleted
I20260809 14:30:08.469508 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 03f6a524d4ef48c4a31e0bfec2f557bb (DEFAULT_TABLE table=default.Table_2c3e9ee14404475bba64491f9d132524 [id=07a8421008714d8eb96d3d29c06c8fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.472904 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 301b53eeb3be41a2b8ff2598d1b531d7
I20260809 14:30:08.474730 31945 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ee3df66954244dd79fb6af9208969ed8" } new_table_name: "default.Table_bb45fcc479b0477f872971659421b3ac" modify_external_catalogs: true
I20260809 14:30:08.477178 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 6fa25dbd78f64e5db9e31d433e8f4736
I20260809 14:30:08.477289 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d7cbb2bb26c4bd7a089648007d7f041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.478391 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03f6a524d4ef48c4a31e0bfec2f557bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.478471 31942 catalog_manager.cc:6565] ReplaceTablet: tablet 6fa25dbd78f64e5db9e31d433e8f4736 deleted and replaced by tablet 66e42e58096c447db3315436c361db0b
I20260809 14:30:08.479053 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5396854feb14153a115f1425f8f4224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.493804 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet 301b53eeb3be41a2b8ff2598d1b531d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:08.495033 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 9e38a13e6e1c480880eab4bc2d2113cd (DEFAULT_TABLE table=default.Table_0227da888bb24888976eae18fce543e9 [id=bb9c3a0b52f0419db0b22211334a0aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.497344 32137 tablet_service.cc:1467] Processing CreateTablet for tablet b5396854feb14153a115f1425f8f4224 (DEFAULT_TABLE table=default.Table_7930652dac8a4ec885cd18e733b1e84e [id=1dd4e785ac184325be3876d8148799f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.498710 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5396854feb14153a115f1425f8f4224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.502317 32304 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fa25dbd78f64e5db9e31d433e8f4736 with delete_type TABLET_DATA_DELETED (replaced by tablet 66e42e58096c447db3315436c361db0b) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:08.504177 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 200c48c11cb042169b307bc1902be82e (DEFAULT_TABLE table=default.Table_65d17735090446738a487e765aef028a [id=6d697fa0067f4a3c960023c24567477c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.505442 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 200c48c11cb042169b307bc1902be82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.509965 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 3d7cbb2bb26c4bd7a089648007d7f041 (DEFAULT_TABLE table=default.Table_e8d440cfb3be48698670be598799bce1 [id=f0ca818a03d640338e363e544463efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.511006 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d7cbb2bb26c4bd7a089648007d7f041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.513931 32141 tablet_service.cc:1467] Processing CreateTablet for tablet 9e38a13e6e1c480880eab4bc2d2113cd (DEFAULT_TABLE table=default.Table_0227da888bb24888976eae18fce543e9 [id=bb9c3a0b52f0419db0b22211334a0aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.508778 32693 tablet_bootstrap.cc:654] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:08.515046 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e38a13e6e1c480880eab4bc2d2113cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.518174 32136 tablet_service.cc:1467] Processing CreateTablet for tablet f06cf519a49441ae885a73e295fb585d (DEFAULT_TABLE table=default.Table_701ce3bbb495413c9497791f1a125753 [id=aa63b376e3864a83b98964845691de6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.519330 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06cf519a49441ae885a73e295fb585d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.520042 608 raft_consensus.cc:482] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:08.520584 608 raft_consensus.cc:504] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.521293 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 3619d49d0b944a91bcb181e884047b38
W20260809 14:29:55.161948 32722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 14:30:08.561666 32304 tablet_service.cc:1514] Processing DeleteTablet for tablet 3619d49d0b944a91bcb181e884047b38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:08.508152 32301 tablet_service.cc:1467] Processing CreateTablet for tablet a0b196e429514cbc803e9a99a986d4d2 (DEFAULT_TABLE table=default.Table_b9541d0208bb4c938aff912a0df84a22 [id=58e17b048a594020a069730291d81345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.562772 579 raft_consensus.cc:482] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:08.563410 579 raft_consensus.cc:504] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.565934 579 leader_election.cc:288] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:29:56.498996 32721 debug-util.cc:398] Leaking SignalData structure 0x7b0800014860 after lost signal to thread 32671
W20260809 14:30:08.573979 32721 kernel_stack_watchdog.cc:198] Thread 32671 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 349ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260809 14:30:08.578943 32727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 14:30:08.580390 32725 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 13419 milliseconds
W20260809 14:30:08.581307 32671 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 13.434s user 0.677s sys 1.386s
I20260809 14:30:08.584784 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 29fdaef8a95e4aa0b23018d384442d16
I20260809 14:30:08.586158 31937 catalog_manager.cc:6565] ReplaceTablet: tablet 29fdaef8a95e4aa0b23018d384442d16 deleted and replaced by tablet 1eb9d7c27381473aa87b718cca57b678
I20260809 14:30:08.586539 31937 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49782 (request call id 233) took 1235 ms. Trace:
W20260809 14:30:08.589547 32671 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 13.442s user 0.677s sys 1.386s
I20260809 14:30:08.590224 32135 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fa25dbd78f64e5db9e31d433e8f4736 with delete_type TABLET_DATA_DELETED (replaced by tablet 66e42e58096c447db3315436c361db0b) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:08.590914 32671 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260809 14:30:08.594091 32671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 14:30:08.586843 31937 rpcz_store.cc:276] 0809 14:30:07.350893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.352548 (+ 1655us) service_pool.cc:227] Handling call
0809 14:30:08.586470 (+1233922us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:08.505356 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:08.505764 (+ 408us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:08.505793 (+ 29us) write_op.cc:435] Acquired schema lock
0809 14:30:08.505809 (+ 16us) tablet.cc:660] Decoding operations
0809 14:30:08.519949 (+ 14140us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:08.519983 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:08.520003 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:08.520211 (+ 208us) tablet.cc:699] Row locks acquired
0809 14:30:08.520223 (+ 12us) write_op.cc:260] PREPARE: finished
0809 14:30:08.520324 (+ 101us) write_op.cc:270] Start()
0809 14:30:08.520399 (+ 75us) write_op.cc:276] Timestamp: P: 1786285808518822 usec, L: 0
0809 14:30:08.520413 (+ 14us) op_driver.cc:331] REPLICATION: starting
0809 14:30:08.520869 (+ 456us) log.cc:842] Serialized 628 byte log entry
0809 14:30:08.579251 (+ 58382us) op_driver.cc:444] REPLICATION: finished
0809 14:30:08.579724 (+ 473us) write_op.cc:301] APPLY: starting
0809 14:30:08.579789 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:08.579920 (+ 131us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:08.580020 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:08.580419 (+ 399us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:08.580684 (+ 265us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:08.580750 (+ 66us) write_op.cc:312] APPLY: finished
0809 14:30:08.580981 (+ 231us) log.cc:842] Serialized 29 byte log entry
0809 14:30:08.581154 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:08.581234 (+ 80us) write_op.cc:454] Released schema lock
0809 14:30:08.581271 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:08.581487 (+ 216us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:08.581521 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":314,"apply.run_cpu_time_us":2014,"apply.run_wall_time_us":1998,"num_ops":2,"prepare.queue_time_us":38518,"prepare.run_cpu_time_us":1886,"prepare.run_wall_time_us":15679,"replication_time_us":58747}]]}
I20260809 14:30:08.597142 31943 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_5fdc5800b255423da5d44ad4ab4a1c2a [id=20d2073f354649b3b4bc4d583660c93a] alter complete (version 3)
I20260809 14:30:08.598538 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e38a13e6e1c480880eab4bc2d2113cd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:08.598577 31943 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 60) took 812 ms (client timeout 1000 ms). Trace:
I20260809 14:30:08.599200 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b196e429514cbc803e9a99a986d4d2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:08.598922 31943 rpcz_store.cc:269] 0809 14:30:07.786148 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.786245 (+ 97us) service_pool.cc:227] Handling call
0809 14:30:08.598522 (+812277us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:08.522805 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:08.523097 (+ 292us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:08.523113 (+ 16us) write_op.cc:435] Acquired schema lock
0809 14:30:08.523123 (+ 10us) tablet.cc:660] Decoding operations
0809 14:30:08.523323 (+ 200us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:08.523351 (+ 28us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:08.523370 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:08.523485 (+ 115us) tablet.cc:699] Row locks acquired
0809 14:30:08.523510 (+ 25us) write_op.cc:260] PREPARE: finished
0809 14:30:08.523593 (+ 83us) write_op.cc:270] Start()
0809 14:30:08.523654 (+ 61us) write_op.cc:276] Timestamp: P: 1786285808522094 usec, L: 0
0809 14:30:08.523669 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 14:30:08.524080 (+ 411us) log.cc:842] Serialized 356 byte log entry
0809 14:30:08.592268 (+ 68188us) op_driver.cc:444] REPLICATION: finished
0809 14:30:08.592578 (+ 310us) write_op.cc:301] APPLY: starting
0809 14:30:08.592625 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:08.592716 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:08.592797 (+ 81us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:08.592934 (+ 137us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:08.593056 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:08.593097 (+ 41us) write_op.cc:312] APPLY: finished
0809 14:30:08.593248 (+ 151us) log.cc:842] Serialized 23 byte log entry
0809 14:30:08.593810 (+ 562us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:08.593883 (+ 73us) write_op.cc:454] Released schema lock
0809 14:30:08.593911 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:08.594064 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:08.594088 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":182,"apply.run_cpu_time_us":1212,"apply.run_wall_time_us":1644,"num_ops":1,"prepare.queue_time_us":42828,"prepare.run_cpu_time_us":1443,"prepare.run_wall_time_us":1443,"replication_time_us":68523}]]}
I20260809 14:30:08.611737 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609b3e6049ec4b0582d78037b7416710" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:08.612257 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet 301b53eeb3be41a2b8ff2598d1b531d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:08.613072 589 raft_consensus.cc:482] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:08.613543 589 raft_consensus.cc:504] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:08.615564 589 leader_election.cc:288] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:08.621769 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b196e429514cbc803e9a99a986d4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.622432 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03f6a524d4ef48c4a31e0bfec2f557bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.623013 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eba8c216630d482db5bf235b4c8f5b15. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:08.623481 31933 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.125:42169, user_credentials={real_user=slave}} blocked reactor thread for 45090.5us
I20260809 14:30:08.612560 32515 raft_consensus.cc:2454] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:08.628619 32671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 14:30:08.642760 608 leader_election.cc:288] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:08.643321 608 raft_consensus.cc:482] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:08.645713 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 1eb9d7c27381473aa87b718cca57b678) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:08.646772 32671 hybrid_clock.cc:648] HybridClock initialized: now 1786285808646655 us; error 116 us; skew 500 ppm
I20260809 14:30:08.647816 32304 tablet_service.cc:1467] Processing CreateTablet for tablet d737c419ce924d2d982474c74019c472 (DEFAULT_TABLE table=default.Table_853333f626c34bc18f5a1245314d7e99 [id=504e8c38c3f1441ebe085ccbcbdab567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.648196 32671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 14:30:08.649224 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d737c419ce924d2d982474c74019c472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.656213 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 29cd0861b04747699ab2d13cb89b418d (DEFAULT_TABLE table=default.Table_b46cfe02c4084ffbb8825b06376fc849 [id=7f0923619d004dcbb14d481d42ca3341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.657519 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cd0861b04747699ab2d13cb89b418d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.638417 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609b3e6049ec4b0582d78037b7416710" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:08.660912 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "810c39c60b1942e2ab1e48624aeb0ee2" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:08.662401 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ad3946f5de4387af3627a88b6404cf" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:08.663319 32326 raft_consensus.cc:2454] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:30:08.665190 32297 tablet_service.cc:1467] Processing CreateTablet for tablet c8a95e406071421cac1704a42a5f78ff (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.666473 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a95e406071421cac1704a42a5f78ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.656220 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 37a3a71874fd44bf865b11c05e84ca84 (DEFAULT_TABLE table=default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.669556 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a3a71874fd44bf865b11c05e84ca84. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:08.673540 32260 leader_election.cc:341] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:08.674216 32260 leader_election.cc:341] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:08.674767 32260 leader_election.cc:302] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:08.675544 589 raft_consensus.cc:2781] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:08.675869 589 raft_consensus.cc:482] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:08.676229 589 raft_consensus.cc:3037] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:08.676239 426 ts_tablet_manager.cc:1918] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:08.676621 426 log.cc:1196] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/810c39c60b1942e2ab1e48624aeb0ee2
I20260809 14:30:08.677250 426 ts_tablet_manager.cc:1939] T 810c39c60b1942e2ab1e48624aeb0ee2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:08.678957 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 119) took 5219 ms. Trace:
I20260809 14:30:08.679234 426 rpcz_store.cc:276] 0809 14:30:03.459397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.464582 (+ 5185us) service_pool.cc:227] Handling call
0809 14:30:08.311600 (+4847018us) ts_tablet_manager.cc:1145] Deleting tablet 810c39c60b1942e2ab1e48624aeb0ee2
0809 14:30:08.311641 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.613584 (+301943us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.676014 (+ 62430us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.678906 (+ 2892us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4845944}
I20260809 14:30:08.680186 426 tablet_replica.cc:330] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:08.682898 31937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3c5c49afb5814baf8e9f3f33f5beac33" } modify_external_catalogs: true
I20260809 14:30:08.683502 31937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3c5c49afb5814baf8e9f3f33f5beac33" } modify_external_catalogs: true
I20260809 14:30:08.690565 353 ts_tablet_manager.cc:1918] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:08.690976 353 log.cc:1196] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/422067edd08745bebdf59d4da704e665
I20260809 14:30:08.691794 353 ts_tablet_manager.cc:1939] T 422067edd08745bebdf59d4da704e665 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:08.694119 32705 tablet_bootstrap.cc:654] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:08.694299 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 146) took 1879 ms. Trace:
I20260809 14:30:08.694657 353 rpcz_store.cc:276] 0809 14:30:06.814700 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:06.814875 (+ 175us) service_pool.cc:227] Handling call
0809 14:30:08.345795 (+1530920us) ts_tablet_manager.cc:1145] Deleting tablet 422067edd08745bebdf59d4da704e665
0809 14:30:08.345849 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.392984 (+ 47135us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.654169 (+261185us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.694230 (+ 40061us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1530335}
I20260809 14:30:08.702477 32139 tablet_service.cc:1467] Processing CreateTablet for tablet d737c419ce924d2d982474c74019c472 (DEFAULT_TABLE table=default.Table_853333f626c34bc18f5a1245314d7e99 [id=504e8c38c3f1441ebe085ccbcbdab567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.703437 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d737c419ce924d2d982474c74019c472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.705166 32135 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 1eb9d7c27381473aa87b718cca57b678) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:08.715931 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 3619d49d0b944a91bcb181e884047b38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:08.722889 626 ts_tablet_manager.cc:1918] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:08.723371 626 log.cc:1196] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/6f36ab4503f24ca2a2a0d812c4c52468
I20260809 14:30:08.723649 589 raft_consensus.cc:504] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.724262 626 ts_tablet_manager.cc:1939] T 6f36ab4503f24ca2a2a0d812c4c52468 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:08.725172 626 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 158) took 1154 ms. Trace:
I20260809 14:30:08.725620 626 rpcz_store.cc:276] 0809 14:30:07.571161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.571766 (+ 605us) service_pool.cc:227] Handling call
0809 14:30:08.303225 (+731459us) ts_tablet_manager.cc:1145] Deleting tablet 6f36ab4503f24ca2a2a0d812c4c52468
0809 14:30:08.303322 (+ 97us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.390935 (+ 87613us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.722630 (+331695us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:08.725116 (+ 2486us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":730746}
I20260809 14:30:08.729662 31946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e8d440cfb3be48698670be598799bce1" } modify_external_catalogs: true
I20260809 14:30:08.730192 31946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e8d440cfb3be48698670be598799bce1" } modify_external_catalogs: true
I20260809 14:30:08.733053 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 37a3a71874fd44bf865b11c05e84ca84 (DEFAULT_TABLE table=default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.738317 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 29cd0861b04747699ab2d13cb89b418d (DEFAULT_TABLE table=default.Table_b46cfe02c4084ffbb8825b06376fc849 [id=7f0923619d004dcbb14d481d42ca3341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.739560 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cd0861b04747699ab2d13cb89b418d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.744107 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e5e6a1c2957d4ebfaa299a5bf66454e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:08.745922 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e5e6a1c2957d4ebfaa299a5bf66454e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:08.740736 32148 tablet_service.cc:1467] Processing CreateTablet for tablet c8a95e406071421cac1704a42a5f78ff (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.755412 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a3a71874fd44bf865b11c05e84ca84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.759742 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a95e406071421cac1704a42a5f78ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.764110 589 leader_election.cc:288] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:08.764365 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 422067edd08745bebdf59d4da704e665 (table default.Table_d200d3ada57d40339da8cd5f8499cabc [id=8d157c6db35a4a6f976fed683f2d1d17]) successfully deleted
I20260809 14:30:08.765111 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 148) took 1790 ms. Trace:
I20260809 14:30:08.765422 353 rpcz_store.cc:276] 0809 14:30:06.974766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:06.974900 (+ 134us) service_pool.cc:227] Handling call
0809 14:30:08.764702 (+1789802us) ts_tablet_manager.cc:1145] Deleting tablet 2af8b48c8cde4f11a53eec939a1f8e3d
0809 14:30:08.764764 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.765058 (+ 294us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1789271}
I20260809 14:30:08.767114 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 149) took 1757 ms. Trace:
I20260809 14:30:08.767490 353 rpcz_store.cc:276] 0809 14:30:07.009676 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.009968 (+ 292us) service_pool.cc:227] Handling call
0809 14:30:08.766704 (+1756736us) ts_tablet_manager.cc:1145] Deleting tablet 9922cad6d10945d5b013a168a3496676
0809 14:30:08.766775 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.767062 (+ 287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1755905}
I20260809 14:30:08.741647 32298 tablet_service.cc:1467] Processing CreateTablet for tablet f06cf519a49441ae885a73e295fb585d (DEFAULT_TABLE table=default.Table_701ce3bbb495413c9497791f1a125753 [id=aa63b376e3864a83b98964845691de6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.750113 32693 tablet_bootstrap.cc:492] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:08.775303 32693 ts_tablet_manager.cc:1397] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.391s user 0.014s sys 0.011s
I20260809 14:30:08.778052 32693 raft_consensus.cc:348] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:08.778846 32693 raft_consensus.cc:374] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:08.779146 32693 raft_consensus.cc:729] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:08.795509 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_ed937fcc529a4bad9edc6085fdf5b47d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:08.796702 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06cf519a49441ae885a73e295fb585d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:08.800608 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed937fcc529a4bad9edc6085fdf5b47d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:08.807689 31943 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
I20260809 14:30:08.810727 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 6f36ab4503f24ca2a2a0d812c4c52468 (table default.Table_259db63a3b974d09ae70ab777de2033f [id=e4e69ddcbdf647cb9e6fd48d47afca31]) successfully deleted
W20260809 14:30:08.811623 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet d4980140ba2a4ecc87001e2a43c8e813 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4980140ba2a4ecc87001e2a43c8e813 already in progress: creating tablet
I20260809 14:30:08.817399 31943 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 2a689385d063469595fcae82aa979bb1 (table default.Table_d841d100269948ca8f7fc467c85a3b0d [id=c19378f8e8994ef28e7b0eba079948c3]). Expected version 2 got 0
I20260809 14:30:08.812120 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 147) took 1859 ms. Trace:
I20260809 14:30:08.819618 353 rpcz_store.cc:276] 0809 14:30:06.952788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.043962 (+ 91174us) service_pool.cc:227] Handling call
0809 14:30:08.811773 (+1767811us) ts_tablet_manager.cc:1145] Deleting tablet 3d592b1c4520467791ffd9d36e0d97fd
0809 14:30:08.811825 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.812068 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1761314}
I20260809 14:30:08.832105 353 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 150) took 1222 ms. Trace:
I20260809 14:30:08.832512 353 rpcz_store.cc:276] 0809 14:30:07.609764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.609969 (+ 205us) service_pool.cc:227] Handling call
0809 14:30:08.831691 (+1221722us) ts_tablet_manager.cc:1145] Deleting tablet d4980140ba2a4ecc87001e2a43c8e813
0809 14:30:08.831749 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:08.832044 (+ 295us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1220916}
I20260809 14:30:08.840597 611 raft_consensus.cc:482] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:08.841081 611 raft_consensus.cc:504] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:08.843711 611 leader_election.cc:288] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:08.855393 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 2af8b48c8cde4f11a53eec939a1f8e3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2af8b48c8cde4f11a53eec939a1f8e3d already in progress: creating tablet
W20260809 14:30:08.856084 31933 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 88646us
I20260809 14:30:08.849356 32693 consensus_queue.cc:260] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:08.643787 608 raft_consensus.cc:504] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.882385 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 9e38a13e6e1c480880eab4bc2d2113cd (DEFAULT_TABLE table=default.Table_0227da888bb24888976eae18fce543e9 [id=bb9c3a0b52f0419db0b22211334a0aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.883100 608 leader_election.cc:288] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:08.883435 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e38a13e6e1c480880eab4bc2d2113cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.884666 32443 leader_election.cc:302] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
W20260809 14:30:08.885303 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 171766us
I20260809 14:30:08.887642 673 raft_consensus.cc:2781] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:08.887969 673 raft_consensus.cc:482] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:08.888235 673 raft_consensus.cc:3037] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:08.888823 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609b3e6049ec4b0582d78037b7416710" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:08.889353 32515 raft_consensus.cc:3037] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:08.892802 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a689385d063469595fcae82aa979bb1" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:08.893460 32326 raft_consensus.cc:2454] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
W20260809 14:30:08.870527 32490 consensus_meta.cc:287] T 868a8000400147c89e9d919c179fd3a0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.504s user 0.008s sys 0.001s
I20260809 14:30:08.897224 32487 tablet_service.cc:1467] Processing CreateTablet for tablet b5396854feb14153a115f1425f8f4224 (DEFAULT_TABLE table=default.Table_7930652dac8a4ec885cd18e733b1e84e [id=1dd4e785ac184325be3876d8148799f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.877238 426 raft_consensus.cc:2229] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:08.879076 32485 tablet_service.cc:1467] Processing CreateTablet for tablet 200c48c11cb042169b307bc1902be82e (DEFAULT_TABLE table=default.Table_65d17735090446738a487e765aef028a [id=6d697fa0067f4a3c960023c24567477c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.880709 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 3d7cbb2bb26c4bd7a089648007d7f041 (DEFAULT_TABLE table=default.Table_e8d440cfb3be48698670be598799bce1 [id=f0ca818a03d640338e363e544463efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.898089 426 raft_consensus.cc:2258] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:08.898988 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d7cbb2bb26c4bd7a089648007d7f041. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:08.697463 32691 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.004s sys 0.008s
I20260809 14:30:08.899653 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5396854feb14153a115f1425f8f4224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.900380 32693 ts_tablet_manager.cc:1428] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.125s user 0.007s sys 0.000s
I20260809 14:30:08.901773 32693 tablet_bootstrap.cc:492] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:08.905788 426 ts_tablet_manager.cc:1905] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:08.899623 32691 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.712s user 0.005s sys 0.013s
I20260809 14:30:08.912904 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 200c48c11cb042169b307bc1902be82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:08.914626 32691 tablet_bootstrap.cc:492] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:08.915093 32691 ts_tablet_manager.cc:1397] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.829s user 0.015s sys 0.014s
I20260809 14:30:08.915457 32693 tablet_bootstrap.cc:654] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:08.916153 32296 tablet_service.cc:1514] Processing DeleteTablet for tablet 609b3e6049ec4b0582d78037b7416710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:08.924759 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 9922cad6d10945d5b013a168a3496676 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9922cad6d10945d5b013a168a3496676 already in progress: creating tablet
I20260809 14:30:08.925382 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:08.926409 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:08.928820 353 tablet_replica.cc:330] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
W20260809 14:30:08.929033 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 3d592b1c4520467791ffd9d36e0d97fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d592b1c4520467791ffd9d36e0d97fd already in progress: creating tablet
W20260809 14:30:08.930850 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet d4980140ba2a4ecc87001e2a43c8e813 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4980140ba2a4ecc87001e2a43c8e813 already in progress: creating tablet
W20260809 14:30:08.933784 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet da7b2e6ca78b4fffb6f8daff1d874afe because the tablet was not found. No further retry: Not found: Tablet not found: da7b2e6ca78b4fffb6f8daff1d874afe
W20260809 14:30:08.943143 32260 leader_election.cc:341] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 810c39c60b1942e2ab1e48624aeb0ee2
I20260809 14:30:08.943575 32260 leader_election.cc:302] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:08.944315 589 raft_consensus.cc:2726] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:08.952915 32443 leader_election.cc:302] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:08.953847 608 raft_consensus.cc:2781] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:08.954190 608 raft_consensus.cc:482] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:08.954505 608 raft_consensus.cc:3037] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:08.944288 31942 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
I20260809 14:30:08.958401 32515 raft_consensus.cc:2454] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
W20260809 14:30:08.961896 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 84c8eb8ad2dd406085779ed6a86eb70e because the tablet was not found. No further retry: Not found: Tablet not found: 84c8eb8ad2dd406085779ed6a86eb70e
I20260809 14:30:08.962473 608 raft_consensus.cc:504] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:08.964845 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 301b53eeb3be41a2b8ff2598d1b531d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 301b53eeb3be41a2b8ff2598d1b531d7 already in progress: creating tablet
I20260809 14:30:08.967046 608 leader_election.cc:288] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:08.968588 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 6fa25dbd78f64e5db9e31d433e8f4736 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6fa25dbd78f64e5db9e31d433e8f4736 already in progress: creating tablet
I20260809 14:30:08.969190 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ad3946f5de4387af3627a88b6404cf" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:08.969422 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 3619d49d0b944a91bcb181e884047b38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3619d49d0b944a91bcb181e884047b38 already in progress: creating tablet
W20260809 14:30:08.970252 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 29fdaef8a95e4aa0b23018d384442d16 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29fdaef8a95e4aa0b23018d384442d16 already in progress: creating tablet
I20260809 14:30:08.975088 32671 webserver.cc:460] Webserver started at http://127.30.252.126:39845/ using document root <none> and password file <none>
I20260809 14:30:08.976665 32671 fs_manager.cc:359] Metadata directory not provided
I20260809 14:30:08.977015 32671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 14:30:08.989289 32481 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fa25dbd78f64e5db9e31d433e8f4736 with delete_type TABLET_DATA_DELETED (replaced by tablet 66e42e58096c447db3315436c361db0b) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:08.989975 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 3619d49d0b944a91bcb181e884047b38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:08.990548 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609b3e6049ec4b0582d78037b7416710" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:08.990813 32479 tablet_service.cc:1467] Processing CreateTablet for tablet f06cf519a49441ae885a73e295fb585d (DEFAULT_TABLE table=default.Table_701ce3bbb495413c9497791f1a125753 [id=aa63b376e3864a83b98964845691de6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:08.992676 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet 301b53eeb3be41a2b8ff2598d1b531d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:08.993333 32691 raft_consensus.cc:348] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:08.994117 32691 raft_consensus.cc:374] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:08.994411 32691 raft_consensus.cc:729] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:08.994459 32260 leader_election.cc:302] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:08.995172 589 raft_consensus.cc:2781] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:08.995693 589 raft_consensus.cc:686] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:08.996532 589 consensus_queue.cc:237] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:09.009068 31938 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 32cb7842617a4779bf110e4ea254e011 consensus state has no leader
W20260809 14:30:09.010403 31945 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b393017627934fa1b740d8ba8ac62d51 consensus state has no leader
I20260809 14:30:09.009532 32490 tablet_service.cc:1467] Processing CreateTablet for tablet d737c419ce924d2d982474c74019c472 (DEFAULT_TABLE table=default.Table_853333f626c34bc18f5a1245314d7e99 [id=504e8c38c3f1441ebe085ccbcbdab567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.010627 32491 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 1eb9d7c27381473aa87b718cca57b678) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:09.012138 32481 tablet_service.cc:1467] Processing CreateTablet for tablet 37a3a71874fd44bf865b11c05e84ca84 (DEFAULT_TABLE table=default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.016476 32477 tablet_service.cc:1467] Processing CreateTablet for tablet c8a95e406071421cac1704a42a5f78ff (DEFAULT_TABLE table=default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.016724 32691 consensus_queue.cc:260] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:09.018909 31930 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 83755.6us
I20260809 14:30:09.019169 32691 ts_tablet_manager.cc:1428] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.104s user 0.006s sys 0.000s
I20260809 14:30:09.019551 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 810c39c60b1942e2ab1e48624aeb0ee2 (table default.Table_e7a7c8e0160944e2847c372937ae5d40 [id=f2ed48a0b8934893874b5fe8d5c6617d]) successfully deleted
I20260809 14:30:09.019698 673 raft_consensus.cc:504] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.014955 32478 tablet_service.cc:1467] Processing CreateTablet for tablet 29cd0861b04747699ab2d13cb89b418d (DEFAULT_TABLE table=default.Table_b46cfe02c4084ffbb8825b06376fc849 [id=7f0923619d004dcbb14d481d42ca3341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.022076 673 leader_election.cc:288] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:09.028923 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06cf519a49441ae885a73e295fb585d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:09.037444 32099 leader_election.cc:341] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 3705cf965d214eb794984b9c16604326
I20260809 14:30:09.044651 353 raft_consensus.cc:2229] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:09.045357 353 pending_rounds.cc:62] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 1 pending ops.
I20260809 14:30:09.045607 353 pending_rounds.cc:68] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626673660547072 op_type: NO_OP noop_request { }
I20260809 14:30:09.046036 353 raft_consensus.cc:2866] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:09.046355 353 raft_consensus.cc:2258] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.049985 589 raft_consensus.cc:482] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:09.050508 589 raft_consensus.cc:504] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:09.053610 589 leader_election.cc:288] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:09.055992 32304 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad3946f5de4387af3627a88b6404cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:09.057583 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f36ab4503f24ca2a2a0d812c4c52468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:09.058203 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 3705cf965d214eb794984b9c16604326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:09.065100 626 tablet_replica.cc:330] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:09.065431 31937 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1eb9d7c27381473aa87b718cca57b678
I20260809 14:30:09.067107 626 raft_consensus.cc:2229] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.074287 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a7a1c8e38748cd9a3c4ccef2aea13c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:09.075109 32515 raft_consensus.cc:2379] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3eb0bc287b12421d89abf97ca66e4d9b in current term 1: Already voted for candidate e1f94e0d792c46c9a0b6ab42820da6eb in this term.
W20260809 14:30:09.073269 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 6fa25dbd78f64e5db9e31d433e8f4736 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6fa25dbd78f64e5db9e31d433e8f4736 already in progress: creating tablet
W20260809 14:30:09.081430 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 118089us
I20260809 14:30:09.084944 32705 tablet_bootstrap.cc:492] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:09.085546 32705 ts_tablet_manager.cc:1397] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.823s user 0.011s sys 0.016s
I20260809 14:30:09.085913 353 ts_tablet_manager.cc:1905] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.088680 32705 raft_consensus.cc:348] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.089713 32705 raft_consensus.cc:374] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.090085 32705 raft_consensus.cc:729] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.090884 32705 consensus_queue.cc:260] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.094794 32705 ts_tablet_manager.cc:1428] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260809 14:30:09.068410 626 raft_consensus.cc:2258] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.099956 626 ts_tablet_manager.cc:1905] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:09.096258 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 301b53eeb3be41a2b8ff2598d1b531d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 301b53eeb3be41a2b8ff2598d1b531d7 already in progress: creating tablet
W20260809 14:30:09.111837 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 3619d49d0b944a91bcb181e884047b38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3619d49d0b944a91bcb181e884047b38 already in progress: creating tablet
W20260809 14:30:09.119491 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 6f36ab4503f24ca2a2a0d812c4c52468 because the tablet was not found. No further retry: Not found: Tablet not found: 6f36ab4503f24ca2a2a0d812c4c52468
I20260809 14:30:09.089043 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a3a71874fd44bf865b11c05e84ca84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.138249 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a95e406071421cac1704a42a5f78ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.104077 32691 tablet_bootstrap.cc:492] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:30:09.141671 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 91974.7us
I20260809 14:30:09.120187 31946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 3d7cbb2bb26c4bd7a089648007d7f041
I20260809 14:30:09.143687 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a689385d063469595fcae82aa979bb1" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:09.144624 32304 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d7cbb2bb26c4bd7a089648007d7f041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:08 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:09.145464 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a689385d063469595fcae82aa979bb1" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:09.139065 31948 master_service.cc:946] ReplaceTablet: received request to replace tablet 273ca6176a184a719538a9455afded4d from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:09.147083 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ad3946f5de4387af3627a88b6404cf" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:09.148644 32439 leader_election.cc:341] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:09.150079 32260 leader_election.cc:341] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.150503 32705 tablet_bootstrap.cc:492] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:09.152859 32439 leader_election.cc:341] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:09.153916 32439 leader_election.cc:341] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:09.154680 32439 leader_election.cc:341] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.161859 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d7cbb2bb26c4bd7a089648007d7f041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:08 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:09.166524 32693 tablet_bootstrap.cc:492] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:09.167021 32693 ts_tablet_manager.cc:1397] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.265s user 0.035s sys 0.030s
I20260809 14:30:09.173993 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cd0861b04747699ab2d13cb89b418d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:09.156555 32095 leader_election.cc:341] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 3705cf965d214eb794984b9c16604326
I20260809 14:30:09.175752 32095 leader_election.cc:302] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:09.175587 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ad3946f5de4387af3627a88b6404cf" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
W20260809 14:30:09.176307 32095 raft_consensus.cc:2670] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:09.176355 32329 raft_consensus.cc:3037] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:09.176920 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a689385d063469595fcae82aa979bb1" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:09.177556 32326 raft_consensus.cc:3037] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:09.181228 32260 leader_election.cc:341] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Not found: Tablet not found: 56a7a1c8e38748cd9a3c4ccef2aea13c
I20260809 14:30:09.181612 32260 leader_election.cc:302] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.181696 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_a7b9b0894c4a47afb6848436134e53fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:09.182725 589 raft_consensus.cc:3037] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:09.183595 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7b9b0894c4a47afb6848436134e53fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.184698 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d737c419ce924d2d982474c74019c472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.192584 589 raft_consensus.cc:2726] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:09.201653 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d7cbb2bb26c4bd7a089648007d7f041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:08 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:09.208884 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 609b3e6049ec4b0582d78037b7416710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:09.217761 32693 raft_consensus.cc:348] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.218439 32693 raft_consensus.cc:374] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.218715 32693 raft_consensus.cc:729] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.234035 32326 raft_consensus.cc:2454] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:09.257114 32443 leader_election.cc:302] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.256947 32693 consensus_queue.cc:260] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.259897 32693 ts_tablet_manager.cc:1428] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.093s user 0.005s sys 0.002s
I20260809 14:30:09.264765 32329 raft_consensus.cc:2454] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:09.265909 673 raft_consensus.cc:2781] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:09.266669 673 raft_consensus.cc:686] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:09.270869 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_647afce4319c40c7ac1de82ead70b0c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:09.275285 31938 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
I20260809 14:30:09.275590 31939 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_6d9b31cc5d9d4976b94613e05d76120c" } new_table_name: "default.Table_7e7ed61d048442e7902bbb1f79677b20" modify_external_catalogs: true
I20260809 14:30:09.277056 31937 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3677aa9bbc744eb59b538d1d53adc9e3" } new_table_name: "default.Table_511f2222eb5843dfae37dae407d870d1" modify_external_catalogs: true
I20260809 14:30:09.273840 426 ts_tablet_manager.cc:1918] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:09.280774 426 log.cc:1196] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/da7b2e6ca78b4fffb6f8daff1d874afe
I20260809 14:30:09.281517 426 ts_tablet_manager.cc:1939] T da7b2e6ca78b4fffb6f8daff1d874afe P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:09.282641 31946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b831dc978eb249f7b28215db2cc26ef0" } modify_external_catalogs: true
I20260809 14:30:09.284183 32691 tablet_bootstrap.cc:654] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.285653 673 consensus_queue.cc:237] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.289000 32671 fs_manager.cc:709] Time spent opening directory manager: real 0.206s user 0.004s sys 0.006s
I20260809 14:30:09.291119 31940 master_service.cc:946] ReplaceTablet: received request to replace tablet d737c419ce924d2d982474c74019c472 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:09.283780 31946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b831dc978eb249f7b28215db2cc26ef0" } modify_external_catalogs: true
I20260809 14:30:09.316699 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 126) took 5191 ms. Trace:
I20260809 14:30:09.317025 426 rpcz_store.cc:276] 0809 14:30:04.125213 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.125372 (+ 159us) service_pool.cc:227] Handling call
0809 14:30:08.680059 (+4554687us) ts_tablet_manager.cc:1145] Deleting tablet da7b2e6ca78b4fffb6f8daff1d874afe
0809 14:30:08.680095 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.071107 (+391012us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.258176 (+187069us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.316640 (+ 58464us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":196205,"tablet-delete.queue_time_us":4552257}
I20260809 14:30:09.319015 426 tablet_replica.cc:330] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:09.320254 426 raft_consensus.cc:2229] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.320724 32705 tablet_bootstrap.cc:654] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.321095 426 raft_consensus.cc:2258] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.318286 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet da7b2e6ca78b4fffb6f8daff1d874afe (table default.Table_3677aa9bbc744eb59b538d1d53adc9e3 [id=2cf0c0e1fb6f40c29bb391b38988e91a]) successfully deleted
W20260809 14:30:09.325817 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_647afce4319c40c7ac1de82ead70b0c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.329466 32443 leader_election.cc:302] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.286471 32298 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a7a1c8e38748cd9a3c4ccef2aea13c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:09.329756 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_385287d0f4f64e54a624637cdb94d2f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:09.330178 608 raft_consensus.cc:2781] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:09.330581 608 raft_consensus.cc:686] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:09.331306 608 consensus_queue.cc:237] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.261410 32693 tablet_bootstrap.cc:492] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:09.316314 353 ts_tablet_manager.cc:1918] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:09.342335 353 log.cc:1196] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/609b3e6049ec4b0582d78037b7416710
I20260809 14:30:09.343258 353 ts_tablet_manager.cc:1939] T 609b3e6049ec4b0582d78037b7416710 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
W20260809 14:30:09.344946 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_385287d0f4f64e54a624637cdb94d2f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.347704 626 ts_tablet_manager.cc:1918] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.348223 626 log.cc:1196] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/3705cf965d214eb794984b9c16604326
I20260809 14:30:09.354364 426 ts_tablet_manager.cc:1905] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.365352 673 raft_consensus.cc:482] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:09.365947 673 raft_consensus.cc:504] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.366799 32691 tablet_bootstrap.cc:492] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:09.367316 32691 ts_tablet_manager.cc:1397] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.264s user 0.017s sys 0.000s
I20260809 14:30:09.368676 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:09.371481 32439 leader_election.cc:341] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.370961 32691 raft_consensus.cc:348] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.372498 32691 raft_consensus.cc:374] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.372611 673 leader_election.cc:288] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:09.372810 32691 raft_consensus.cc:729] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.373642 32691 consensus_queue.cc:260] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.378708 32691 ts_tablet_manager.cc:1428] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.011s user 0.003s sys 0.003s
I20260809 14:30:09.381554 32691 tablet_bootstrap.cc:492] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:09.382225 690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 14:30:09.384827 353 tablet_replica.cc:330] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:09.385241 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 609b3e6049ec4b0582d78037b7416710 (table default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]) successfully deleted
I20260809 14:30:09.385638 353 raft_consensus.cc:2229] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.389004 626 ts_tablet_manager.cc:1939] T 3705cf965d214eb794984b9c16604326 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:09.390534 32671 fs_manager.cc:725] Time spent opening block manager: real 0.067s user 0.001s sys 0.004s
I20260809 14:30:09.394992 673 raft_consensus.cc:482] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:30:09.395584 31939 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d885baba6d6045e8b1ebcab6513719bb consensus state has no leader
I20260809 14:30:09.395532 673 raft_consensus.cc:504] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.386194 353 raft_consensus.cc:2258] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.397435 673 leader_election.cc:288] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:09.398254 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:09.399912 32439 leader_election.cc:341] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.401154 32691 tablet_bootstrap.cc:654] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.404048 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c2edbb238dd341b49bcd5acde282fb46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:09.407629 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_77a290e4278b410cbfbe30feb015ef83" } modify_external_catalogs: true
I20260809 14:30:09.408115 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_77a290e4278b410cbfbe30feb015ef83" } modify_external_catalogs: true
W20260809 14:30:09.408802 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2edbb238dd341b49bcd5acde282fb46 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.405793 353 ts_tablet_manager.cc:1905] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.414374 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:09.415962 32443 leader_election.cc:341] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.416360 32443 leader_election.cc:302] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.417732 608 raft_consensus.cc:2726] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:09.418303 31937 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 37a3a71874fd44bf865b11c05e84ca84 consensus state has no leader
I20260809 14:30:09.390988 32671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data,/tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
uuid: "9d10512ac8b3406098fad1f7da012ab3"
format_stamp: "Formatted at 2026-08-09 14:29:39 on dist-test-slave-10pc"
I20260809 14:30:09.419699 426 ts_tablet_manager.cc:1918] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.420035 426 log.cc:1196] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/609b3e6049ec4b0582d78037b7416710
I20260809 14:30:09.420707 426 ts_tablet_manager.cc:1939] T 609b3e6049ec4b0582d78037b7416710 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:09.422590 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 127) took 5229 ms. Trace:
I20260809 14:30:09.422910 426 rpcz_store.cc:276] 0809 14:30:04.193052 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.193252 (+ 200us) service_pool.cc:227] Handling call
0809 14:30:09.318757 (+5125505us) ts_tablet_manager.cc:1145] Deleting tablet 609b3e6049ec4b0582d78037b7416710
0809 14:30:09.318827 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.405077 (+ 86250us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.419485 (+ 14408us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.422536 (+ 3051us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5124589}
I20260809 14:30:09.424165 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 609b3e6049ec4b0582d78037b7416710 (table default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]) successfully deleted
I20260809 14:30:09.424876 426 tablet_replica.cc:330] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:09.424587 32671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 14:30:09.425628 426 raft_consensus.cc:2229] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.426039 426 raft_consensus.cc:2258] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.427811 32691 tablet_bootstrap.cc:492] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:09.428328 32691 ts_tablet_manager.cc:1397] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.049s user 0.011s sys 0.001s
I20260809 14:30:09.432525 426 ts_tablet_manager.cc:1905] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.433498 32691 raft_consensus.cc:348] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.434130 32691 raft_consensus.cc:374] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.434378 32691 raft_consensus.cc:729] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.435173 32691 consensus_queue.cc:260] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.439504 32693 tablet_bootstrap.cc:654] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.392642 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:09.444106 32691 ts_tablet_manager.cc:1428] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
W20260809 14:30:09.445264 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b393017627934fa1b740d8ba8ac62d51 (table default.Table_bb45fcc479b0477f872971659421b3ac [id=7043672a284145c7a5c9a5d80c322ef8], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet b393017627934fa1b740d8ba8ac62d51 consensus state has no leader
I20260809 14:30:09.446815 32691 tablet_bootstrap.cc:492] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:30:09.447599 32443 leader_election.cc:341] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.447978 32443 leader_election.cc:302] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.449347 673 raft_consensus.cc:2726] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:09.451802 32691 tablet_bootstrap.cc:654] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.461022 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 3705cf965d214eb794984b9c16604326 (table default.Table_f7fd4627a85c4f358f737fad5077415c [id=12165d071bb546219c54224908e47280]) successfully deleted
I20260809 14:30:09.461649 32705 tablet_bootstrap.cc:492] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:09.462076 32705 ts_tablet_manager.cc:1397] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.312s user 0.004s sys 0.020s
W20260809 14:30:09.462716 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 3d7cbb2bb26c4bd7a089648007d7f041 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d7cbb2bb26c4bd7a089648007d7f041 already in progress: creating tablet
I20260809 14:30:09.462785 426 ts_tablet_manager.cc:1918] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.463110 426 log.cc:1196] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:09.463703 426 ts_tablet_manager.cc:1939] T 07ac3ef5573c40b894c9b2e743ca0582 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:09.464288 626 tablet_replica.cc:330] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:09.465595 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 128) took 5272 ms. Trace:
I20260809 14:30:09.465937 426 rpcz_store.cc:276] 0809 14:30:04.193589 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.194401 (+ 812us) service_pool.cc:227] Handling call
0809 14:30:09.424733 (+5230332us) ts_tablet_manager.cc:1145] Deleting tablet 07ac3ef5573c40b894c9b2e743ca0582
0809 14:30:09.424770 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.456057 (+ 31287us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.462581 (+ 6524us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.465536 (+ 2955us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5229710}
I20260809 14:30:09.466621 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_18d2bdbeb7ad42e9b0edf59e2e89e8b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:09.467278 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 07ac3ef5573c40b894c9b2e743ca0582 (table default.Table_abacee8a40094093b6e3454fedfa8ca6 [id=329d73d7b2244b42805b4b7cd0e2e45c]) successfully deleted
W20260809 14:30:09.468003 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18d2bdbeb7ad42e9b0edf59e2e89e8b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.473054 426 tablet_replica.cc:330] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:09.473788 426 raft_consensus.cc:2229] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.474272 426 raft_consensus.cc:2258] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.475561 32693 tablet_bootstrap.cc:492] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:09.476052 32693 ts_tablet_manager.cc:1397] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.215s user 0.013s sys 0.000s
I20260809 14:30:09.478986 626 raft_consensus.cc:2229] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.479422 426 ts_tablet_manager.cc:1905] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.478946 32693 raft_consensus.cc:348] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.479540 626 raft_consensus.cc:2258] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.479807 32693 raft_consensus.cc:374] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.465668 32705 raft_consensus.cc:348] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:09.480103 32693 raft_consensus.cc:729] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.480512 32705 raft_consensus.cc:374] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.480801 32693 consensus_queue.cc:260] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.486253 32705 raft_consensus.cc:729] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.487003 32705 consensus_queue.cc:260] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:09.496320 32705 ts_tablet_manager.cc:1428] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.034s user 0.005s sys 0.000s
I20260809 14:30:09.499147 32691 tablet_bootstrap.cc:492] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:09.499656 32691 ts_tablet_manager.cc:1397] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.053s user 0.019s sys 0.001s
I20260809 14:30:09.508963 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 810c39c60b1942e2ab1e48624aeb0ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:29:58 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:09.510435 32693 ts_tablet_manager.cc:1428] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260809 14:30:09.511080 32691 raft_consensus.cc:348] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.511864 32691 raft_consensus.cc:374] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.514247 32693 tablet_bootstrap.cc:492] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:09.517570 32705 tablet_bootstrap.cc:492] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:09.524220 32691 raft_consensus.cc:729] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.525123 32691 consensus_queue.cc:260] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.529834 32691 ts_tablet_manager.cc:1428] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.030s user 0.006s sys 0.001s
I20260809 14:30:09.530725 32691 tablet_bootstrap.cc:492] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:09.482777 626 ts_tablet_manager.cc:1905] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.540206 426 ts_tablet_manager.cc:1918] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:09.540628 426 log.cc:1196] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/849d798165144531810a03dc2b9b5f1e
I20260809 14:30:09.541360 426 ts_tablet_manager.cc:1939] T 849d798165144531810a03dc2b9b5f1e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:09.543342 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 129) took 4917 ms. Trace:
I20260809 14:30:09.543668 426 rpcz_store.cc:276] 0809 14:30:04.625751 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.625913 (+ 162us) service_pool.cc:227] Handling call
0809 14:30:09.472889 (+4846976us) ts_tablet_manager.cc:1145] Deleting tablet 849d798165144531810a03dc2b9b5f1e
0809 14:30:09.472940 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.522009 (+ 49069us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.539999 (+ 17990us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.543289 (+ 3290us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4846422}
I20260809 14:30:09.544907 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 849d798165144531810a03dc2b9b5f1e (table default.Table_3228a1bca97b4dfbbbcb8095a1da792e [id=45f3914d4c214f1587b0bea324c32a6c]) successfully deleted
I20260809 14:30:09.556072 32691 tablet_bootstrap.cc:654] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.557209 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 130) took 4838 ms. Trace:
I20260809 14:30:09.559639 32705 tablet_bootstrap.cc:654] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.557576 426 rpcz_store.cc:276] 0809 14:30:04.718999 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:04.719141 (+ 142us) service_pool.cc:227] Handling call
0809 14:30:09.556824 (+4837683us) ts_tablet_manager.cc:1145] Deleting tablet faf222c49c934d52a2642f20e8575702
0809 14:30:09.556870 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.557151 (+ 281us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4837060}
W20260809 14:30:09.561707 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet faf222c49c934d52a2642f20e8575702 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet faf222c49c934d52a2642f20e8575702 already in progress: creating tablet
I20260809 14:30:09.562052 353 ts_tablet_manager.cc:1918] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.562561 353 log.cc:1196] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/810c39c60b1942e2ab1e48624aeb0ee2
I20260809 14:30:09.562865 426 tablet_replica.cc:330] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:09.563326 353 ts_tablet_manager.cc:1939] T 810c39c60b1942e2ab1e48624aeb0ee2 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:09.563716 426 raft_consensus.cc:2229] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.564272 426 raft_consensus.cc:2258] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.568065 32693 tablet_bootstrap.cc:654] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.576926 673 raft_consensus.cc:482] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:09.577526 673 raft_consensus.cc:504] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.581568 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ee617d858c42f7b03bd287a33eebe4" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:09.581746 673 leader_election.cc:288] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:09.582788 32443 leader_election.cc:341] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.582401 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ee617d858c42f7b03bd287a33eebe4" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:09.584612 32439 leader_election.cc:341] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.585023 32439 leader_election.cc:302] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.585773 673 raft_consensus.cc:2726] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:09.587549 426 ts_tablet_manager.cc:1905] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.593210 32691 tablet_bootstrap.cc:492] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:09.593667 32691 ts_tablet_manager.cc:1397] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.007s
W20260809 14:30:09.598981 31930 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111) [suppressed 5 similar messages]
I20260809 14:30:09.600209 31945 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f00ff0058556436f9c41b2b6532452b1" } new_table_name: "default.Table_e329e3111b054b7db1bdeb7e4a91720d" modify_external_catalogs: true
I20260809 14:30:09.604743 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 810c39c60b1942e2ab1e48624aeb0ee2 (table default.Table_e7a7c8e0160944e2847c372937ae5d40 [id=f2ed48a0b8934893874b5fe8d5c6617d]) successfully deleted
I20260809 14:30:09.605578 589 raft_consensus.cc:482] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:09.606308 589 raft_consensus.cc:504] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.610858 32691 raft_consensus.cc:348] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.611621 32691 raft_consensus.cc:374] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.611927 32691 raft_consensus.cc:729] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.611765 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:09.613538 32260 leader_election.cc:341] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.614811 32691 consensus_queue.cc:260] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.615764 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:09.616482 32515 raft_consensus.cc:2454] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:09.617667 32691 ts_tablet_manager.cc:1428] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.024s user 0.007s sys 0.000s
I20260809 14:30:09.618767 32260 leader_election.cc:302] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.619453 579 raft_consensus.cc:2781] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:09.619786 579 raft_consensus.cc:482] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:09.619545 589 leader_election.cc:288] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:09.620136 579 raft_consensus.cc:3037] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:09.605633 353 tablet_replica.cc:330] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:09.621461 353 raft_consensus.cc:2229] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.621927 353 raft_consensus.cc:2258] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.625417 426 ts_tablet_manager.cc:1918] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:09.625836 426 log.cc:1196] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/84c8eb8ad2dd406085779ed6a86eb70e
I20260809 14:30:09.625892 31937 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_90fd7293a898493897859b855282cbbc" } new_table_name: "default.Table_506db6ae9cbd4fe7b4973e9eb2b71136" modify_external_catalogs: true
I20260809 14:30:09.626645 426 ts_tablet_manager.cc:1939] T 84c8eb8ad2dd406085779ed6a86eb70e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:09.628065 32691 tablet_bootstrap.cc:492] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:09.633237 32693 tablet_bootstrap.cc:492] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:09.633790 32693 ts_tablet_manager.cc:1397] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.120s user 0.009s sys 0.005s
I20260809 14:30:09.638206 353 ts_tablet_manager.cc:1905] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.639043 32691 tablet_bootstrap.cc:654] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.645089 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 132) took 4313 ms. Trace:
I20260809 14:30:09.645545 426 rpcz_store.cc:276] 0809 14:30:05.331659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.331807 (+ 148us) service_pool.cc:227] Handling call
0809 14:30:09.562690 (+4230883us) ts_tablet_manager.cc:1145] Deleting tablet 84c8eb8ad2dd406085779ed6a86eb70e
0809 14:30:09.562739 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.614125 (+ 51386us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.625181 (+ 11056us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.645023 (+ 19842us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4166579}
I20260809 14:30:09.646878 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 84c8eb8ad2dd406085779ed6a86eb70e (table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]) successfully deleted
I20260809 14:30:09.647995 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 134) took 4075 ms. Trace:
I20260809 14:30:09.648293 426 rpcz_store.cc:276] 0809 14:30:05.572911 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.574433 (+ 1522us) service_pool.cc:227] Handling call
0809 14:30:09.647655 (+4073222us) ts_tablet_manager.cc:1145] Deleting tablet 4cdfc56edcd34696a7b806b0ee34d594
0809 14:30:09.647698 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.647943 (+ 245us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4072193}
I20260809 14:30:09.649573 32705 tablet_bootstrap.cc:492] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
W20260809 14:30:09.649518 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 4cdfc56edcd34696a7b806b0ee34d594 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cdfc56edcd34696a7b806b0ee34d594 already in progress: creating tablet
I20260809 14:30:09.649958 32705 ts_tablet_manager.cc:1397] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.133s user 0.008s sys 0.002s
I20260809 14:30:09.650434 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 135) took 4077 ms. Trace:
I20260809 14:30:09.650720 426 rpcz_store.cc:276] 0809 14:30:05.573574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.575795 (+ 2221us) service_pool.cc:227] Handling call
0809 14:30:09.650214 (+4074419us) ts_tablet_manager.cc:1145] Deleting tablet 969849d30c204937a7508e8d54dae427
0809 14:30:09.650249 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.650387 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4073125}
W20260809 14:30:09.651908 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 969849d30c204937a7508e8d54dae427 because the tablet was not found. No further retry: Not found: Tablet not found: 969849d30c204937a7508e8d54dae427
I20260809 14:30:09.652765 426 tablet_replica.cc:330] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:09.652707 32705 raft_consensus.cc:348] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.653491 32705 raft_consensus.cc:374] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.653755 32705 raft_consensus.cc:729] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.653513 32693 raft_consensus.cc:348] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.654271 32693 raft_consensus.cc:374] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.654521 32693 raft_consensus.cc:729] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.654551 32705 consensus_queue.cc:260] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.655798 32691 tablet_bootstrap.cc:492] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:09.656275 32691 ts_tablet_manager.cc:1397] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.028s user 0.014s sys 0.001s
I20260809 14:30:09.657683 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_feb15cc2cdf64b3bb4c6de7992d27e89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:09.659308 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_feb15cc2cdf64b3bb4c6de7992d27e89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.659781 32691 raft_consensus.cc:348] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.660497 32691 raft_consensus.cc:374] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.660734 32691 raft_consensus.cc:729] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.661381 32691 consensus_queue.cc:260] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.663496 426 raft_consensus.cc:2229] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.663899 426 raft_consensus.cc:2258] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.665903 426 ts_tablet_manager.cc:1905] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.675386 32691 ts_tablet_manager.cc:1428] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.019s user 0.003s sys 0.003s
I20260809 14:30:09.676345 32691 tablet_bootstrap.cc:492] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:30:09.679003 31945 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5634154c6f149d3b2d36b961aee38e0 consensus state has no leader
I20260809 14:30:09.662946 32693 consensus_queue.cc:260] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.683902 32693 ts_tablet_manager.cc:1428] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.050s user 0.005s sys 0.003s
I20260809 14:30:09.693192 426 ts_tablet_manager.cc:1918] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.693564 426 log.cc:1196] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/56a7a1c8e38748cd9a3c4ccef2aea13c
I20260809 14:30:09.694161 426 ts_tablet_manager.cc:1939] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:09.696079 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 138) took 4019 ms. Trace:
I20260809 14:30:09.698638 32693 tablet_bootstrap.cc:492] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:09.702853 32691 tablet_bootstrap.cc:654] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.696347 426 rpcz_store.cc:276] 0809 14:30:05.676781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.677102 (+ 321us) service_pool.cc:227] Handling call
0809 14:30:09.652610 (+3975508us) ts_tablet_manager.cc:1145] Deleting tablet 56a7a1c8e38748cd9a3c4ccef2aea13c
0809 14:30:09.652661 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.684204 (+ 31543us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.692978 (+ 8774us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.696024 (+ 3046us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9977,"tablet-delete.queue_time_us":3974193}
I20260809 14:30:09.710182 32705 ts_tablet_manager.cc:1428] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.060s user 0.007s sys 0.000s
I20260809 14:30:09.711009 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 56a7a1c8e38748cd9a3c4ccef2aea13c (table default.Table_f3f904c0b94440ddb1f6f7f733689d8c [id=ae6b2507dcdd4367b1faa278e84fbe3a]) successfully deleted
I20260809 14:30:09.712015 426 tablet_replica.cc:330] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:09.712563 353 ts_tablet_manager.cc:1918] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.712962 353 log.cc:1196] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/6f36ab4503f24ca2a2a0d812c4c52468
I20260809 14:30:09.713395 426 raft_consensus.cc:2229] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.713665 353 ts_tablet_manager.cc:1939] T 6f36ab4503f24ca2a2a0d812c4c52468 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:09.713953 426 raft_consensus.cc:2258] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.715819 426 ts_tablet_manager.cc:1905] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.716210 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 6f36ab4503f24ca2a2a0d812c4c52468 (table default.Table_259db63a3b974d09ae70ab777de2033f [id=e4e69ddcbdf647cb9e6fd48d47afca31]) successfully deleted
I20260809 14:30:09.723798 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
I20260809 14:30:09.724246 32141 tablet_service.cc:1467] Processing CreateTablet for tablet f5ed582793904bedb6b917677d79d8fa (DEFAULT_TABLE table=default.Table_9c8d59829d3a4c3fa74ab227cb412935 [id=c6717be3432145588399b56c1bd72ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.725996 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ed582793904bedb6b917677d79d8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.732257 32478 tablet_service.cc:1467] Processing CreateTablet for tablet f5ed582793904bedb6b917677d79d8fa (DEFAULT_TABLE table=default.Table_9c8d59829d3a4c3fa74ab227cb412935 [id=c6717be3432145588399b56c1bd72ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.734009 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ed582793904bedb6b917677d79d8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.736941 353 tablet_replica.cc:330] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:09.736593 32304 tablet_service.cc:1467] Processing CreateTablet for tablet e455d9c37f99458a84720427851cd836 (DEFAULT_TABLE table=default.Table_f4fdd6821b4749b4869ebd018c1e7ed7 [id=f2c816cbb179423caf8afd00b900d7b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.737794 353 raft_consensus.cc:2229] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.738304 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e455d9c37f99458a84720427851cd836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.738416 353 raft_consensus.cc:2258] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.739388 32135 tablet_service.cc:1467] Processing CreateTablet for tablet e455d9c37f99458a84720427851cd836 (DEFAULT_TABLE table=default.Table_f4fdd6821b4749b4869ebd018c1e7ed7 [id=f2c816cbb179423caf8afd00b900d7b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.741237 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e455d9c37f99458a84720427851cd836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.742950 32693 tablet_bootstrap.cc:654] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.737439 32298 tablet_service.cc:1467] Processing CreateTablet for tablet f5ed582793904bedb6b917677d79d8fa (DEFAULT_TABLE table=default.Table_9c8d59829d3a4c3fa74ab227cb412935 [id=c6717be3432145588399b56c1bd72ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.745359 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ed582793904bedb6b917677d79d8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.748232 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 62e1825a7642459085f82e584c8a4df8 (DEFAULT_TABLE table=default.Table_6b3c6d27f787472981f7207081957411 [id=f16c8747b93a4544900b7dc94d49af9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.749337 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e1825a7642459085f82e584c8a4df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.751652 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 2a689385d063469595fcae82aa979bb1 (table default.Table_d841d100269948ca8f7fc467c85a3b0d [id=c19378f8e8994ef28e7b0eba079948c3]). Expected version 2 got 0
I20260809 14:30:09.753347 32490 tablet_service.cc:1467] Processing CreateTablet for tablet e455d9c37f99458a84720427851cd836 (DEFAULT_TABLE table=default.Table_f4fdd6821b4749b4869ebd018c1e7ed7 [id=f2c816cbb179423caf8afd00b900d7b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.754070 353 ts_tablet_manager.cc:1905] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.754500 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e455d9c37f99458a84720427851cd836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.755772 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 62e1825a7642459085f82e584c8a4df8 (DEFAULT_TABLE table=default.Table_6b3c6d27f787472981f7207081957411 [id=f16c8747b93a4544900b7dc94d49af9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.757496 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e1825a7642459085f82e584c8a4df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.759929 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:09.767900 32705 tablet_bootstrap.cc:492] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:09.775895 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_478fce1bf3d54075835a5d464370d13e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:09.777489 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_478fce1bf3d54075835a5d464370d13e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.779428 32481 tablet_service.cc:1467] Processing CreateTablet for tablet 62e1825a7642459085f82e584c8a4df8 (DEFAULT_TABLE table=default.Table_6b3c6d27f787472981f7207081957411 [id=f16c8747b93a4544900b7dc94d49af9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.779968 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_7077d276f84a485c840fded026cca74a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:09.781078 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e1825a7642459085f82e584c8a4df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.786926 579 raft_consensus.cc:504] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.791647 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 23aa8db34e704f7aa56acb6af46be5bc (DEFAULT_TABLE table=default.Table_753e14e7053644a3a98b60a13a6c9a66 [id=e5a8b0d198b3475695134c497bb993b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.793160 32136 tablet_service.cc:1467] Processing CreateTablet for tablet 23aa8db34e704f7aa56acb6af46be5bc (DEFAULT_TABLE table=default.Table_753e14e7053644a3a98b60a13a6c9a66 [id=e5a8b0d198b3475695134c497bb993b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.797338 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23aa8db34e704f7aa56acb6af46be5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.802213 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23aa8db34e704f7aa56acb6af46be5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.788116 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_bc907356c8584717b336afc1a260018e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:09.804214 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc907356c8584717b336afc1a260018e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.811719 626 ts_tablet_manager.cc:1918] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.812335 626 log.cc:1196] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/609b3e6049ec4b0582d78037b7416710
I20260809 14:30:09.813194 626 ts_tablet_manager.cc:1939] T 609b3e6049ec4b0582d78037b7416710 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:09.816403 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 609b3e6049ec4b0582d78037b7416710 (table default.Table_c58e6c8a0da84a3eaf25216d4c3d6a2f [id=104d9a5015094e82a8cac7ea3b45c0fc]) successfully deleted
I20260809 14:30:09.820590 589 raft_consensus.cc:482] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:09.821229 589 raft_consensus.cc:504] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.823875 589 leader_election.cc:288] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:09.826314 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ac3ef5573c40b894c9b2e743ca0582" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:09.829543 32260 leader_election.cc:341] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:09.830598 32260 leader_election.cc:341] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:09.830952 32260 leader_election.cc:302] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.832127 589 raft_consensus.cc:2726] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:09.836233 32477 tablet_service.cc:1467] Processing CreateTablet for tablet 23aa8db34e704f7aa56acb6af46be5bc (DEFAULT_TABLE table=default.Table_753e14e7053644a3a98b60a13a6c9a66 [id=e5a8b0d198b3475695134c497bb993b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.838554 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23aa8db34e704f7aa56acb6af46be5bc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:09.781271 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7077d276f84a485c840fded026cca74a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:09.846556 353 ts_tablet_manager.cc:1918] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.849275 32691 tablet_bootstrap.cc:492] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:09.849766 32691 ts_tablet_manager.cc:1397] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.174s user 0.011s sys 0.010s
W20260809 14:30:09.855443 31943 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 43) took 800 ms (client timeout 1000 ms). Trace:
W20260809 14:30:09.855818 31943 rpcz_store.cc:269] 0809 14:30:09.054694 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:09.062757 (+ 8063us) service_pool.cc:227] Handling call
0809 14:30:09.855388 (+792631us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":8}
I20260809 14:30:09.861346 32479 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:09.862151 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad3946f5de4387af3627a88b6404cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:09.864084 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:09.865687 32260 leader_election.cc:341] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:09.867549 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 86702ef8331241d486a1f82609b5de83 (DEFAULT_TABLE table=default.Table_3c7d49528e7746e7b4711c684fe4191e [id=7e86056e9bdd48fca9442015a748a495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.869122 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86702ef8331241d486a1f82609b5de83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.877570 353 log.cc:1196] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/b0ad3946f5de4387af3627a88b6404cf
I20260809 14:30:09.878433 353 ts_tablet_manager.cc:1939] T b0ad3946f5de4387af3627a88b6404cf P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:09.883559 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:09.884320 32515 raft_consensus.cc:3037] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:09.892201 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 86702ef8331241d486a1f82609b5de83 (DEFAULT_TABLE table=default.Table_3c7d49528e7746e7b4711c684fe4191e [id=7e86056e9bdd48fca9442015a748a495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.893810 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86702ef8331241d486a1f82609b5de83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.893364 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 86702ef8331241d486a1f82609b5de83 (DEFAULT_TABLE table=default.Table_3c7d49528e7746e7b4711c684fe4191e [id=7e86056e9bdd48fca9442015a748a495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.896943 32691 raft_consensus.cc:348] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.897675 32691 raft_consensus.cc:374] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.897950 32691 raft_consensus.cc:729] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:09.898557 32691 consensus_queue.cc:260] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.898870 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86702ef8331241d486a1f82609b5de83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.902225 32515 raft_consensus.cc:2454] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:09.903638 32260 leader_election.cc:302] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:09.904706 589 raft_consensus.cc:2781] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:09.905603 32691 ts_tablet_manager.cc:1428] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.056s user 0.005s sys 0.002s
I20260809 14:30:09.906585 32691 tablet_bootstrap.cc:492] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:09.914072 32705 tablet_bootstrap.cc:654] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:09.917340 426 ts_tablet_manager.cc:1918] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:09.917726 426 log.cc:1196] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/6f36ab4503f24ca2a2a0d812c4c52468
I20260809 14:30:09.920435 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 23d2b217278c481e8464d00188c70048 (DEFAULT_TABLE table=default.Table_45d346a083d04fc6b8a9d8631c65bff1 [id=4a72c13a29f74ba19c69e2392c122b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.921698 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d2b217278c481e8464d00188c70048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.922385 32137 tablet_service.cc:1467] Processing CreateTablet for tablet ac27f377aa824654b6491c3065ef819c (DEFAULT_TABLE table=default.Table_b831dc978eb249f7b28215db2cc26ef0 [id=32f53ee209704b6ca8117dc7c937e6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.924137 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac27f377aa824654b6491c3065ef819c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.924115 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 23d2b217278c481e8464d00188c70048 (DEFAULT_TABLE table=default.Table_45d346a083d04fc6b8a9d8631c65bff1 [id=4a72c13a29f74ba19c69e2392c122b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.925143 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d2b217278c481e8464d00188c70048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.926265 32693 tablet_bootstrap.cc:492] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:09.926657 32693 ts_tablet_manager.cc:1397] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.228s user 0.014s sys 0.012s
I20260809 14:30:09.933282 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet b0ad3946f5de4387af3627a88b6404cf (table default.Table_bb02de675fd247e5af1ff74a6c57f2bd [id=eb07bcad144c4719ae97a233970493f8]) successfully deleted
W20260809 14:30:09.934783 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 3d7cbb2bb26c4bd7a089648007d7f041 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d7cbb2bb26c4bd7a089648007d7f041 already in progress: creating tablet
I20260809 14:30:09.935772 353 tablet_replica.cc:330] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:09.936506 353 raft_consensus.cc:2229] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.936951 353 raft_consensus.cc:2258] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.938709 353 ts_tablet_manager.cc:1905] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:09.939560 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 23d2b217278c481e8464d00188c70048 (DEFAULT_TABLE table=default.Table_45d346a083d04fc6b8a9d8631c65bff1 [id=4a72c13a29f74ba19c69e2392c122b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.940970 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d2b217278c481e8464d00188c70048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.937105 426 ts_tablet_manager.cc:1939] T 6f36ab4503f24ca2a2a0d812c4c52468 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:09.946470 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 144) took 4024 ms. Trace:
I20260809 14:30:09.946817 426 rpcz_store.cc:276] 0809 14:30:05.922683 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.922840 (+ 157us) service_pool.cc:227] Handling call
0809 14:30:09.711835 (+3788995us) ts_tablet_manager.cc:1145] Deleting tablet 6f36ab4503f24ca2a2a0d812c4c52468
0809 14:30:09.711883 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.866875 (+154992us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.917120 (+ 50245us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:09.946407 (+ 29287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3787723}
I20260809 14:30:09.948153 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 143) took 4026 ms. Trace:
I20260809 14:30:09.948340 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 6f36ab4503f24ca2a2a0d812c4c52468 (table default.Table_259db63a3b974d09ae70ab777de2033f [id=e4e69ddcbdf647cb9e6fd48d47afca31]) successfully deleted
I20260809 14:30:09.955582 32298 tablet_service.cc:1467] Processing CreateTablet for tablet ac27f377aa824654b6491c3065ef819c (DEFAULT_TABLE table=default.Table_b831dc978eb249f7b28215db2cc26ef0 [id=32f53ee209704b6ca8117dc7c937e6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.960839 426 rpcz_store.cc:276] 0809 14:30:05.922076 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.923648 (+ 1572us) service_pool.cc:227] Handling call
0809 14:30:09.947874 (+4024226us) ts_tablet_manager.cc:1145] Deleting tablet 01cfc9e0319748c8b0e928fc64706933
0809 14:30:09.947924 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:09.948098 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4023495}
I20260809 14:30:09.956480 579 leader_election.cc:288] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:09.962098 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 01cfc9e0319748c8b0e928fc64706933 because the tablet was not found. No further retry: Not found: Tablet not found: 01cfc9e0319748c8b0e928fc64706933
I20260809 14:30:09.967082 426 tablet_replica.cc:330] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:09.967993 426 raft_consensus.cc:2229] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:09.968806 426 raft_consensus.cc:2258] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:09.972965 32478 tablet_service.cc:1467] Processing CreateTablet for tablet ac27f377aa824654b6491c3065ef819c (DEFAULT_TABLE table=default.Table_b831dc978eb249f7b28215db2cc26ef0 [id=32f53ee209704b6ca8117dc7c937e6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.973213 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 04b74ba6259e47569d13dfe62d14a4af (DEFAULT_TABLE table=default.Table_533f32539e9542569c81b5bfe1f0f53e [id=237c0b3d040a41f39b4a8b9344933a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.974833 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac27f377aa824654b6491c3065ef819c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:09.977870 31937 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 428a3e63b617400ca3747abc64321618 consensus state has no leader
I20260809 14:30:09.977880 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 04b74ba6259e47569d13dfe62d14a4af (DEFAULT_TABLE table=default.Table_533f32539e9542569c81b5bfe1f0f53e [id=237c0b3d040a41f39b4a8b9344933a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:09.982018 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04b74ba6259e47569d13dfe62d14a4af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.981454 589 raft_consensus.cc:686] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:09.985565 589 consensus_queue.cc:237] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.988003 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac27f377aa824654b6491c3065ef819c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.981081 32693 raft_consensus.cc:348] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:09.989430 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04b74ba6259e47569d13dfe62d14a4af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:09.989589 32693 raft_consensus.cc:374] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:09.990094 32693 raft_consensus.cc:729] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.012658 626 tablet_replica.cc:330] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:10.019850 32481 tablet_service.cc:1467] Processing CreateTablet for tablet 04b74ba6259e47569d13dfe62d14a4af (DEFAULT_TABLE table=default.Table_533f32539e9542569c81b5bfe1f0f53e [id=237c0b3d040a41f39b4a8b9344933a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.020341 32693 consensus_queue.cc:260] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.020845 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 21530b16a1f941f7a54c8e736c2b79ef (DEFAULT_TABLE table=default.Table_60564937e4494c0ca93832cb7817a76c [id=074d8baf9d43478cb631d61a71d58225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.022306 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21530b16a1f941f7a54c8e736c2b79ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.024365 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04b74ba6259e47569d13dfe62d14a4af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.026187 673 raft_consensus.cc:482] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:10.031684 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_511f2222eb5843dfae37dae407d870d1" } new_table_name: "default.Table_61b1d3ac3b1f4314b31339c66858eb78" modify_external_catalogs: true
I20260809 14:30:10.037175 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_513f49ccff6d4f15bd52a9814a93e513"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:10.038410 32691 tablet_bootstrap.cc:654] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:10.039233 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_513f49ccff6d4f15bd52a9814a93e513 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:10.041051 426 ts_tablet_manager.cc:1905] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:10.054080 32295 tablet_service.cc:1467] Processing CreateTablet for tablet 428a3e63b617400ca3747abc64321618 (DEFAULT_TABLE table=default.Table_506db6ae9cbd4fe7b4973e9eb2b71136 [id=9850853f450545ca81cac41c0eecd0df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.054271 31944 catalog_manager.cc:5495] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 1 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } }
I20260809 14:30:10.055837 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428a3e63b617400ca3747abc64321618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.056005 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 2a689385d063469595fcae82aa979bb1 (table default.Table_d841d100269948ca8f7fc467c85a3b0d [id=c19378f8e8994ef28e7b0eba079948c3]). Expected version 2 got 0
I20260809 14:30:10.080514 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]). Expected version 2 got 0
I20260809 14:30:10.081285 32479 tablet_service.cc:1467] Processing CreateTablet for tablet 428a3e63b617400ca3747abc64321618 (DEFAULT_TABLE table=default.Table_506db6ae9cbd4fe7b4973e9eb2b71136 [id=9850853f450545ca81cac41c0eecd0df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.081480 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:10.082443 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428a3e63b617400ca3747abc64321618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.082134 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:10.081285 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 21530b16a1f941f7a54c8e736c2b79ef (DEFAULT_TABLE table=default.Table_60564937e4494c0ca93832cb7817a76c [id=074d8baf9d43478cb631d61a71d58225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.086109 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21530b16a1f941f7a54c8e736c2b79ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.101533 31948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 273ca6176a184a719538a9455afded4d
I20260809 14:30:10.103001 31948 catalog_manager.cc:6565] ReplaceTablet: tablet 273ca6176a184a719538a9455afded4d deleted and replaced by tablet 8c06535e3b5e441c89e9db7431267f98
I20260809 14:30:10.106340 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 273ca6176a184a719538a9455afded4d with delete_type TABLET_DATA_DELETED (replaced by tablet 8c06535e3b5e441c89e9db7431267f98) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:10.106506 426 ts_tablet_manager.cc:1918] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:10.106842 426 log.cc:1196] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/422067edd08745bebdf59d4da704e665
I20260809 14:30:10.107931 426 ts_tablet_manager.cc:1939] T 422067edd08745bebdf59d4da704e665 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:10.022954 32136 tablet_service.cc:1467] Processing CreateTablet for tablet 21530b16a1f941f7a54c8e736c2b79ef (DEFAULT_TABLE table=default.Table_60564937e4494c0ca93832cb7817a76c [id=074d8baf9d43478cb631d61a71d58225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.109288 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21530b16a1f941f7a54c8e736c2b79ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.109964 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 145) took 4186 ms. Trace:
I20260809 14:30:10.051455 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 428a3e63b617400ca3747abc64321618 (DEFAULT_TABLE table=default.Table_506db6ae9cbd4fe7b4973e9eb2b71136 [id=9850853f450545ca81cac41c0eecd0df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.063133 32705 tablet_bootstrap.cc:492] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:10.105273 32135 tablet_service.cc:1514] Processing DeleteTablet for tablet 273ca6176a184a719538a9455afded4d with delete_type TABLET_DATA_DELETED (replaced by tablet 8c06535e3b5e441c89e9db7431267f98) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:10.112993 32705 ts_tablet_manager.cc:1397] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.345s user 0.014s sys 0.001s
I20260809 14:30:10.113139 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428a3e63b617400ca3747abc64321618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.110282 426 rpcz_store.cc:276] 0809 14:30:05.923334 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:05.924504 (+ 1170us) service_pool.cc:227] Handling call
0809 14:30:09.966887 (+4042383us) ts_tablet_manager.cc:1145] Deleting tablet 422067edd08745bebdf59d4da704e665
0809 14:30:09.966945 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.092706 (+125761us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:10.106287 (+ 13581us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:10.109903 (+ 3616us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4041776}
I20260809 14:30:10.115204 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 151) took 3099 ms. Trace:
I20260809 14:30:10.115379 32705 raft_consensus.cc:348] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.115536 426 rpcz_store.cc:276] 0809 14:30:07.015858 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.016196 (+ 338us) service_pool.cc:227] Handling call
0809 14:30:10.114824 (+3098628us) ts_tablet_manager.cc:1145] Deleting tablet 3d592b1c4520467791ffd9d36e0d97fd
0809 14:30:10.114874 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.115151 (+ 277us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3098007}
I20260809 14:30:10.116216 32705 raft_consensus.cc:374] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.116521 32705 raft_consensus.cc:729] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.116756 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 152) took 3089 ms. Trace:
I20260809 14:30:10.117076 426 rpcz_store.cc:276] 0809 14:30:07.027057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.027271 (+ 214us) service_pool.cc:227] Handling call
0809 14:30:10.116396 (+3089125us) ts_tablet_manager.cc:1145] Deleting tablet 2af8b48c8cde4f11a53eec939a1f8e3d
0809 14:30:10.116439 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.116709 (+ 270us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3088443}
I20260809 14:30:10.117228 32705 consensus_queue.cc:260] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.026686 673 raft_consensus.cc:504] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.119874 673 leader_election.cc:288] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:10.121111 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 273ca6176a184a719538a9455afded4d with delete_type TABLET_DATA_DELETED (replaced by tablet 8c06535e3b5e441c89e9db7431267f98) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:10.121711 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 422067edd08745bebdf59d4da704e665 (table default.Table_d200d3ada57d40339da8cd5f8499cabc [id=8d157c6db35a4a6f976fed683f2d1d17]) successfully deleted
W20260809 14:30:10.122437 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 3d592b1c4520467791ffd9d36e0d97fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d592b1c4520467791ffd9d36e0d97fd already in progress: creating tablet
W20260809 14:30:10.123106 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 2af8b48c8cde4f11a53eec939a1f8e3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2af8b48c8cde4f11a53eec939a1f8e3d already in progress: creating tablet
I20260809 14:30:10.123004 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:10.133353 626 raft_consensus.cc:2229] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:10.134756 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:10.136044 32439 leader_election.cc:341] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.136673 32135 tablet_service.cc:1467] Processing CreateTablet for tablet f43385579b29417289f3bdacf8fce278 (DEFAULT_TABLE table=default.Table_bea4b6049d234a58a4bc8222fd20f462 [id=2a97745346124303a9dce29dbef6727c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:10.137077 32443 leader_election.cc:341] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.137400 32443 leader_election.cc:302] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:10.137722 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f43385579b29417289f3bdacf8fce278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.138880 673 raft_consensus.cc:2726] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:10.149855 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 153) took 3094 ms. Trace:
I20260809 14:30:10.150218 426 rpcz_store.cc:276] 0809 14:30:07.055641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.055840 (+ 199us) service_pool.cc:227] Handling call
0809 14:30:10.117923 (+3062083us) ts_tablet_manager.cc:1145] Deleting tablet 9922cad6d10945d5b013a168a3496676
0809 14:30:10.117964 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.149786 (+ 31822us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3044575}
W20260809 14:30:10.151424 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 9922cad6d10945d5b013a168a3496676 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9922cad6d10945d5b013a168a3496676 already in progress: creating tablet
I20260809 14:30:10.152948 32693 ts_tablet_manager.cc:1428] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.226s user 0.016s sys 0.005s
I20260809 14:30:10.154006 32693 tablet_bootstrap.cc:492] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:10.156782 32307 tablet_service.cc:1467] Processing CreateTablet for tablet f43385579b29417289f3bdacf8fce278 (DEFAULT_TABLE table=default.Table_bea4b6049d234a58a4bc8222fd20f462 [id=2a97745346124303a9dce29dbef6727c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.158030 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f43385579b29417289f3bdacf8fce278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.170410 32691 tablet_bootstrap.cc:492] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:10.170836 32691 ts_tablet_manager.cc:1397] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.264s user 0.012s sys 0.011s
I20260809 14:30:10.173879 32691 raft_consensus.cc:348] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.174592 32691 raft_consensus.cc:374] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.174870 32691 raft_consensus.cc:729] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.174958 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 9c890a8c73044f8ba433151c4efe17af (DEFAULT_TABLE table=default.Table_22a408c4e94c43a29e31198ad59a5e1b [id=ff2c2c4e4811464b97e4bf61226ca6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.175920 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c890a8c73044f8ba433151c4efe17af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.175503 32691 consensus_queue.cc:260] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.177089 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 154) took 2392 ms. Trace:
I20260809 14:30:10.177377 426 rpcz_store.cc:276] 0809 14:30:07.785146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:07.785300 (+ 154us) service_pool.cc:227] Handling call
0809 14:30:10.176738 (+2391438us) ts_tablet_manager.cc:1145] Deleting tablet d4980140ba2a4ecc87001e2a43c8e813
0809 14:30:10.176778 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.177036 (+ 258us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2372939}
I20260809 14:30:10.181705 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 168) took 1192 ms. Trace:
I20260809 14:30:10.182014 426 rpcz_store.cc:276] 0809 14:30:08.989025 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:08.989138 (+ 113us) service_pool.cc:227] Handling call
0809 14:30:10.178122 (+1188984us) ts_tablet_manager.cc:1145] Deleting tablet 6fa25dbd78f64e5db9e31d433e8f4736
0809 14:30:10.181394 (+ 3272us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.181650 (+ 256us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1188385}
I20260809 14:30:10.183090 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 169) took 1193 ms. Trace:
I20260809 14:30:10.183441 426 rpcz_store.cc:276] 0809 14:30:08.989326 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:08.989848 (+ 522us) service_pool.cc:227] Handling call
0809 14:30:10.182785 (+1192937us) ts_tablet_manager.cc:1145] Deleting tablet 3619d49d0b944a91bcb181e884047b38
0809 14:30:10.182821 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.183043 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1192465}
I20260809 14:30:10.184633 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 167) took 1249 ms. Trace:
I20260809 14:30:10.184914 426 rpcz_store.cc:276] 0809 14:30:08.988597 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:08.992551 (+ 3954us) service_pool.cc:227] Handling call
0809 14:30:10.184301 (+1191750us) ts_tablet_manager.cc:1145] Deleting tablet 301b53eeb3be41a2b8ff2598d1b531d7
0809 14:30:10.184335 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.184578 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1191250}
I20260809 14:30:10.185956 426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 172) took 1175 ms. Trace:
I20260809 14:30:10.186218 426 rpcz_store.cc:276] 0809 14:30:09.010337 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:09.010454 (+ 117us) service_pool.cc:227] Handling call
0809 14:30:10.185672 (+1175218us) ts_tablet_manager.cc:1145] Deleting tablet 29fdaef8a95e4aa0b23018d384442d16
0809 14:30:10.185707 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.185901 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1169563}
I20260809 14:30:10.186632 32693 tablet_bootstrap.cc:654] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.188030 426 tablet_replica.cc:330] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:10.194041 32691 ts_tablet_manager.cc:1428] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.023s user 0.004s sys 0.004s
I20260809 14:30:10.201993 32691 tablet_bootstrap.cc:492] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:10.206985 353 ts_tablet_manager.cc:1918] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:10.207346 353 log.cc:1196] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/56a7a1c8e38748cd9a3c4ccef2aea13c
I20260809 14:30:10.207736 32691 tablet_bootstrap.cc:654] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.208020 353 ts_tablet_manager.cc:1939] T 56a7a1c8e38748cd9a3c4ccef2aea13c P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:10.210836 426 raft_consensus.cc:2229] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:10.211616 426 pending_rounds.cc:62] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Trying to abort 1 pending ops.
I20260809 14:30:10.211860 426 pending_rounds.cc:68] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626675031494656 op_type: NO_OP noop_request { }
I20260809 14:30:10.212224 426 raft_consensus.cc:2866] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:10.212560 426 raft_consensus.cc:2258] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:10.216955 426 ts_tablet_manager.cc:1905] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:10.218037 32705 ts_tablet_manager.cc:1428] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.105s user 0.001s sys 0.005s
I20260809 14:30:10.083074 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:10.220897 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_423b0a16be834083833ae85a541ff1c4 [id=c1935d3bc49c405abe4d006932820224]). Expected version 2 got 0
I20260809 14:30:10.221694 32693 tablet_bootstrap.cc:492] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:10.222141 32693 ts_tablet_manager.cc:1397] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.006s
I20260809 14:30:10.226567 626 raft_consensus.cc:2258] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:10.229724 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 56a7a1c8e38748cd9a3c4ccef2aea13c (table default.Table_f3f904c0b94440ddb1f6f7f733689d8c [id=ae6b2507dcdd4367b1faa278e84fbe3a]) successfully deleted
I20260809 14:30:10.231236 353 tablet_replica.cc:330] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:10.232677 353 raft_consensus.cc:2229] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:10.238909 32691 tablet_bootstrap.cc:492] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:10.240880 353 raft_consensus.cc:2258] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:10.242640 353 ts_tablet_manager.cc:1905] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:10.241201 32691 ts_tablet_manager.cc:1397] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.002s
I20260809 14:30:10.246037 32691 raft_consensus.cc:348] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.247350 32691 raft_consensus.cc:374] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.248742 32691 raft_consensus.cc:729] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.249362 32691 consensus_queue.cc:260] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.251549 32691 ts_tablet_manager.cc:1428] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260809 14:30:10.249179 32693 raft_consensus.cc:348] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.252539 32691 tablet_bootstrap.cc:492] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:10.252585 32693 raft_consensus.cc:374] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.253238 32693 raft_consensus.cc:729] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.260874 32693 consensus_queue.cc:260] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.264902 32693 ts_tablet_manager.cc:1428] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.042s user 0.006s sys 0.002s
I20260809 14:30:10.266908 426 ts_tablet_manager.cc:1918] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:10.267272 426 log.cc:1196] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/b0ad3946f5de4387af3627a88b6404cf
I20260809 14:30:10.267966 426 ts_tablet_manager.cc:1939] T b0ad3946f5de4387af3627a88b6404cf P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:10.273300 32693 tablet_bootstrap.cc:492] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:10.275367 32691 tablet_bootstrap.cc:654] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.305521 32705 tablet_bootstrap.cc:492] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:10.309623 32693 tablet_bootstrap.cc:654] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.311738 626 ts_tablet_manager.cc:1905] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:10.326910 353 ts_tablet_manager.cc:1918] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:10.327344 353 log.cc:1196] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:10.328047 353 ts_tablet_manager.cc:1939] T 07ac3ef5573c40b894c9b2e743ca0582 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:10.333055 32297 tablet_service.cc:1467] Processing CreateTablet for tablet e0acb71107a74d178b3512df1ca2f140 (DEFAULT_TABLE table=default.Table_fb1e9afadc124d26b601aeaaaf0d14b3 [id=ca824009fd984674bad52b95f8ba3617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.335033 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0acb71107a74d178b3512df1ca2f140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.343207 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 07ac3ef5573c40b894c9b2e743ca0582 (table default.Table_abacee8a40094093b6e3454fedfa8ca6 [id=329d73d7b2244b42805b4b7cd0e2e45c]) successfully deleted
W20260809 14:30:10.347496 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 273ca6176a184a719538a9455afded4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 273ca6176a184a719538a9455afded4d already in progress: creating tablet
I20260809 14:30:10.313673 32304 tablet_service.cc:1467] Processing CreateTablet for tablet 9c890a8c73044f8ba433151c4efe17af (DEFAULT_TABLE table=default.Table_22a408c4e94c43a29e31198ad59a5e1b [id=ff2c2c4e4811464b97e4bf61226ca6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.353013 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c890a8c73044f8ba433151c4efe17af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.364317 32693 tablet_bootstrap.cc:492] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:10.364753 32693 ts_tablet_manager.cc:1397] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.092s user 0.010s sys 0.008s
I20260809 14:30:10.365464 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_701ce3bbb495413c9497791f1a125753" } new_table_name: "default.Table_3f84f78f94f744ae9a10e20f3b6248ef" modify_external_catalogs: true
I20260809 14:30:10.368086 32693 raft_consensus.cc:348] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.368803 32693 raft_consensus.cc:374] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.369073 32693 raft_consensus.cc:729] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.369666 32693 consensus_queue.cc:260] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.314352 32147 tablet_service.cc:1467] Processing CreateTablet for tablet e0acb71107a74d178b3512df1ca2f140 (DEFAULT_TABLE table=default.Table_fb1e9afadc124d26b601aeaaaf0d14b3 [id=ca824009fd984674bad52b95f8ba3617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.379525 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0acb71107a74d178b3512df1ca2f140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.382942 32693 ts_tablet_manager.cc:1428] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260809 14:30:10.390085 32693 tablet_bootstrap.cc:492] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:10.390955 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet eba8c216630d482db5bf235b4c8f5b15
I20260809 14:30:10.392340 31942 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 264) took 1042 ms. Trace:
I20260809 14:30:10.392665 31942 rpcz_store.cc:276] 0809 14:30:09.350237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:09.350392 (+ 155us) service_pool.cc:227] Handling call
0809 14:30:09.407542 (+ 57150us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9466 00007ffa4b01ab34 00007ffa4b01aac2 00007ffa4b01a9f1 00007ffa4b01a96c 00007ffa4b019c22 00007ffa492b894c 00007ffa492b7d86
0809 14:30:09.410605 (+ 3063us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:09.411097 (+ 492us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:09.411217 (+ 120us) catalog_manager.cc:2761] Locking tablets
0809 14:30:09.733316 (+322099us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:10.390564 (+657248us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:10.390767 (+ 203us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:10.390875 (+ 108us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:10.392286 (+ 1411us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:10.182396 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:10.182886 (+ 490us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:10.182928 (+ 42us) write_op.cc:435] Acquired schema lock
0809 14:30:10.182952 (+ 24us) tablet.cc:660] Decoding operations
0809 14:30:10.183344 (+ 392us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:10.183385 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:10.183404 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:10.183851 (+ 447us) tablet.cc:699] Row locks acquired
0809 14:30:10.183866 (+ 15us) write_op.cc:260] PREPARE: finished
0809 14:30:10.183981 (+ 115us) write_op.cc:270] Start()
0809 14:30:10.184063 (+ 82us) write_op.cc:276] Timestamp: P: 1786285810183913 usec, L: 0
0809 14:30:10.184170 (+ 107us) op_driver.cc:331] REPLICATION: starting
0809 14:30:10.360116 (+175946us) log.cc:842] Serialized 731 byte log entry
0809 14:30:10.360997 (+ 881us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48e471 00007ffa4b505198 00007ffa4b966ade 00007ffa4b9662b8 00007ffa4b96aba1 00007ffa4b96ab59 00007ffa4b96aae9 00007ffa4b96aab1 00007ffa4b969dad 0000000000580784 00000000005805b9
0809 14:30:10.388717 (+ 27720us) op_driver.cc:444] REPLICATION: finished
0809 14:30:10.389004 (+ 287us) write_op.cc:301] APPLY: starting
0809 14:30:10.389061 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:10.389167 (+ 106us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:10.389185 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:10.389421 (+ 236us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:10.389567 (+ 146us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:10.389608 (+ 41us) write_op.cc:312] APPLY: finished
0809 14:30:10.389770 (+ 162us) log.cc:842] Serialized 29 byte log entry
0809 14:30:10.389929 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:10.389989 (+ 60us) write_op.cc:454] Released schema lock
0809 14:30:10.390019 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:10.390195 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:10.390222 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":129391872,"child_traces":[["op",{"apply.queue_time_us":134,"apply.run_cpu_time_us":1389,"apply.run_wall_time_us":1397,"num_ops":2,"prepare.queue_time_us":447044,"prepare.run_cpu_time_us":4531,"prepare.run_wall_time_us":178792,"raft.queue_time_us":7683,"raft.run_cpu_time_us":1213,"raft.run_wall_time_us":1212,"replication_time_us":204475,"spinlock_wait_cycles":403305344}]]}
I20260809 14:30:10.400786 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet eba8c216630d482db5bf235b4c8f5b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:09 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:10.403927 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet eba8c216630d482db5bf235b4c8f5b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:09 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:10.405813 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet eba8c216630d482db5bf235b4c8f5b15 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eba8c216630d482db5bf235b4c8f5b15 already in progress: creating tablet
I20260809 14:30:10.411075 31946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet ac27f377aa824654b6491c3065ef819c
I20260809 14:30:10.412703 31946 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 261) took 1144 ms. Trace:
I20260809 14:30:10.414144 32693 tablet_bootstrap.cc:654] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.413033 31946 rpcz_store.cc:276] 0809 14:30:09.268277 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:09.269654 (+ 1377us) service_pool.cc:227] Handling call
0809 14:30:09.304140 (+ 34486us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:09.307956 (+ 3816us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:09.309016 (+ 1060us) catalog_manager.cc:2761] Locking tablets
0809 14:30:09.726138 (+417122us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:10.410575 (+684437us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:10.410848 (+ 273us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:10.410979 (+ 131us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:10.412642 (+ 1663us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:10.361305 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:10.361675 (+ 370us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:10.361699 (+ 24us) write_op.cc:435] Acquired schema lock
0809 14:30:10.361712 (+ 13us) tablet.cc:660] Decoding operations
0809 14:30:10.362016 (+ 304us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:10.362050 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:10.362072 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:10.362324 (+ 252us) tablet.cc:699] Row locks acquired
0809 14:30:10.362338 (+ 14us) write_op.cc:260] PREPARE: finished
0809 14:30:10.362451 (+ 113us) write_op.cc:270] Start()
0809 14:30:10.362533 (+ 82us) write_op.cc:276] Timestamp: P: 1786285810362407 usec, L: 0
0809 14:30:10.362560 (+ 27us) op_driver.cc:331] REPLICATION: starting
0809 14:30:10.363039 (+ 479us) log.cc:842] Serialized 731 byte log entry
0809 14:30:10.408836 (+ 45797us) op_driver.cc:444] REPLICATION: finished
0809 14:30:10.409135 (+ 299us) write_op.cc:301] APPLY: starting
0809 14:30:10.409203 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:10.409318 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:10.409419 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:10.409673 (+ 254us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:10.409820 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:10.409867 (+ 47us) write_op.cc:312] APPLY: finished
0809 14:30:10.410092 (+ 225us) log.cc:842] Serialized 29 byte log entry
0809 14:30:10.410248 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:10.410317 (+ 69us) write_op.cc:454] Released schema lock
0809 14:30:10.410351 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":90,"num_ops":2,"prepare.queue_time_us":613839,"prepare.run_cpu_time_us":2007,"prepare.run_wall_time_us":9160,"raft.queue_time_us":6952,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":14,"replication_time_us":46040}]]}
I20260809 14:30:10.427842 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet ac27f377aa824654b6491c3065ef819c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:09 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:10.428645 685 raft_consensus.cc:482] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:10.429261 685 raft_consensus.cc:504] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.432260 685 leader_election.cc:288] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:10.434113 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:10.434996 32329 raft_consensus.cc:2454] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:10.436666 32099 leader_election.cc:302] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:10.437677 32705 tablet_bootstrap.cc:654] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.438176 685 raft_consensus.cc:2781] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:10.439229 685 raft_consensus.cc:482] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:10.440148 685 raft_consensus.cc:3037] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:10.440507 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet ac27f377aa824654b6491c3065ef819c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:09 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:10.449047 673 raft_consensus.cc:482] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:30:10.449875 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet ac27f377aa824654b6491c3065ef819c because the tablet was not found. No further retry: Not found: Tablet not found: ac27f377aa824654b6491c3065ef819c
I20260809 14:30:10.450229 673 raft_consensus.cc:504] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.456956 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5121e9f5a85a45d285d78f41a442769a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:10.459347 32439 leader_election.cc:341] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.460718 673 leader_election.cc:288] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:10.466672 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet d737c419ce924d2d982474c74019c472
I20260809 14:30:10.468146 31940 catalog_manager.cc:6565] ReplaceTablet: tablet d737c419ce924d2d982474c74019c472 deleted and replaced by tablet ed328e6f95544c25b08075018cb8caf2
I20260809 14:30:10.468858 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet d737c419ce924d2d982474c74019c472 with delete_type TABLET_DATA_DELETED (replaced by tablet ed328e6f95544c25b08075018cb8caf2) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:10.468446 31940 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49782 (request call id 262) took 1182 ms. Trace:
W20260809 14:30:10.470564 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet d737c419ce924d2d982474c74019c472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d737c419ce924d2d982474c74019c472 already in progress: creating tablet
I20260809 14:30:10.469980 31940 rpcz_store.cc:276] 0809 14:30:09.286503 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:09.290993 (+ 4490us) service_pool.cc:227] Handling call
0809 14:30:10.468398 (+1177405us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:10.370573 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:10.370885 (+ 312us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:10.370907 (+ 22us) write_op.cc:435] Acquired schema lock
0809 14:30:10.370919 (+ 12us) tablet.cc:660] Decoding operations
0809 14:30:10.371196 (+ 277us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:10.371231 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:10.371253 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:10.371422 (+ 169us) tablet.cc:699] Row locks acquired
0809 14:30:10.371435 (+ 13us) write_op.cc:260] PREPARE: finished
0809 14:30:10.371537 (+ 102us) write_op.cc:270] Start()
0809 14:30:10.371629 (+ 92us) write_op.cc:276] Timestamp: P: 1786285810371493 usec, L: 0
0809 14:30:10.371649 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 14:30:10.372064 (+ 415us) log.cc:842] Serialized 628 byte log entry
0809 14:30:10.464157 (+ 92093us) op_driver.cc:444] REPLICATION: finished
0809 14:30:10.464414 (+ 257us) write_op.cc:301] APPLY: starting
0809 14:30:10.464504 (+ 90us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:10.464635 (+ 131us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:10.464747 (+ 112us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:10.465261 (+ 514us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:10.465482 (+ 221us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:10.465534 (+ 52us) write_op.cc:312] APPLY: finished
0809 14:30:10.465750 (+ 216us) log.cc:842] Serialized 29 byte log entry
0809 14:30:10.465908 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:10.465980 (+ 72us) write_op.cc:454] Released schema lock
0809 14:30:10.466014 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":91,"num_ops":2,"prepare.queue_time_us":611066,"prepare.run_cpu_time_us":1668,"prepare.run_wall_time_us":1667,"replication_time_us":92399}]]}
I20260809 14:30:10.474614 31940 master_service.cc:946] ReplaceTablet: received request to replace tablet f06cf519a49441ae885a73e295fb585d from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:10.475469 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet d737c419ce924d2d982474c74019c472 with delete_type TABLET_DATA_DELETED (replaced by tablet ed328e6f95544c25b08075018cb8caf2) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:10.485453 719 raft_consensus.cc:482] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:10.485984 719 raft_consensus.cc:504] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.490223 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6ebda27b44048b5734e8798a980c2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:10.491895 32439 leader_election.cc:341] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.492755 32705 tablet_bootstrap.cc:492] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:10.493147 32705 ts_tablet_manager.cc:1397] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.188s user 0.008s sys 0.002s
I20260809 14:30:10.495208 685 raft_consensus.cc:504] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.488296 719 leader_election.cc:288] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:10.503696 32691 tablet_bootstrap.cc:492] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:10.504333 32691 ts_tablet_manager.cc:1397] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.252s user 0.006s sys 0.017s
I20260809 14:30:10.505493 673 raft_consensus.cc:482] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:10.506057 673 raft_consensus.cc:504] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.506294 31946 catalog_manager.cc:5495] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:10.508831 32691 raft_consensus.cc:348] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.509773 32691 raft_consensus.cc:374] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.510035 32691 raft_consensus.cc:729] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.510675 32691 consensus_queue.cc:260] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.516346 32298 tablet_service.cc:1467] Processing CreateTablet for tablet 66e42e58096c447db3315436c361db0b (DEFAULT_TABLE table=default.Table_76064ebb420d48dfb1586d58bf81d6a3 [id=9a90fee262334c4881f9202da51858ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.517823 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e42e58096c447db3315436c361db0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.505342 32693 tablet_bootstrap.cc:492] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:10.524891 32693 ts_tablet_manager.cc:1397] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.135s user 0.008s sys 0.004s
I20260809 14:30:10.525437 32671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 14:30:10.527477 32671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 14:30:10.527227 32693 raft_consensus.cc:348] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.527863 32693 raft_consensus.cc:374] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.528031 32671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 14:30:10.528124 32693 raft_consensus.cc:729] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.495229 32705 raft_consensus.cc:348] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.528633 32705 raft_consensus.cc:374] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.528903 32705 raft_consensus.cc:729] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.529506 32705 consensus_queue.cc:260] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.508049 626 ts_tablet_manager.cc:1918] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:10.509747 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc8d0cad33b4778af9fe9247743c38f" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:10.530942 626 log.cc:1196] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/810c39c60b1942e2ab1e48624aeb0ee2
I20260809 14:30:10.518532 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 1d52f4b7d49544109df36c085e4aaec3 (DEFAULT_TABLE table=default.Table_250ec22f736f4ef885234ffff62f2054 [id=1ea46aca8f884ea28bebea0f1c9006a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.531704 626 ts_tablet_manager.cc:1939] T 810c39c60b1942e2ab1e48624aeb0ee2 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:10.532078 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d52f4b7d49544109df36c085e4aaec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.533744 626 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 186) took 1025 ms. Trace:
I20260809 14:30:10.534057 626 rpcz_store.cc:276] 0809 14:30:09.508639 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:09.508794 (+ 155us) service_pool.cc:227] Handling call
0809 14:30:10.012345 (+503551us) ts_tablet_manager.cc:1145] Deleting tablet 810c39c60b1942e2ab1e48624aeb0ee2
0809 14:30:10.012416 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:10.482765 (+470349us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:10.507830 (+ 25065us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:10.533687 (+ 25857us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":92375,"tablet-delete.queue_time_us":502923}
I20260809 14:30:10.535435 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 810c39c60b1942e2ab1e48624aeb0ee2 (table default.Table_e7a7c8e0160944e2847c372937ae5d40 [id=f2ed48a0b8934893874b5fe8d5c6617d]) successfully deleted
W20260809 14:30:10.537137 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 273ca6176a184a719538a9455afded4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 273ca6176a184a719538a9455afded4d already in progress: creating tablet
W20260809 14:30:10.539430 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet eba8c216630d482db5bf235b4c8f5b15 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eba8c216630d482db5bf235b4c8f5b15 already in progress: creating tablet
W20260809 14:30:10.541456 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet ac27f377aa824654b6491c3065ef819c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac27f377aa824654b6491c3065ef819c already in progress: creating tablet
W20260809 14:30:10.543635 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet d737c419ce924d2d982474c74019c472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d737c419ce924d2d982474c74019c472 already in progress: creating tablet
I20260809 14:30:10.518659 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 66e42e58096c447db3315436c361db0b (DEFAULT_TABLE table=default.Table_76064ebb420d48dfb1586d58bf81d6a3 [id=9a90fee262334c4881f9202da51858ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.545372 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e42e58096c447db3315436c361db0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.548147 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:10.548794 32329 raft_consensus.cc:3037] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:10.549067 685 leader_election.cc:288] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:10.512310 32691 ts_tablet_manager.cc:1428] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
W20260809 14:30:10.556730 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d885baba6d6045e8b1ebcab6513719bb consensus state has no leader
I20260809 14:30:10.561123 32329 raft_consensus.cc:2454] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:10.562177 32705 ts_tablet_manager.cc:1428] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.069s user 0.009s sys 0.005s
I20260809 14:30:10.562543 32099 leader_election.cc:302] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:10.563374 32705 tablet_bootstrap.cc:492] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:10.568274 735 raft_consensus.cc:2781] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
W20260809 14:30:10.570472 32439 leader_election.cc:341] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.570709 735 raft_consensus.cc:686] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
W20260809 14:30:10.571264 32439 consensus_peers.cc:487] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:10.571961 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 1d52f4b7d49544109df36c085e4aaec3 (DEFAULT_TABLE table=default.Table_250ec22f736f4ef885234ffff62f2054 [id=1ea46aca8f884ea28bebea0f1c9006a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.573154 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d52f4b7d49544109df36c085e4aaec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.576282 32705 tablet_bootstrap.cc:654] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.508296 673 leader_election.cc:288] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:10.580868 735 consensus_queue.cc:237] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.584108 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc8d0cad33b4778af9fe9247743c38f" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:10.586122 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6ebda27b44048b5734e8798a980c2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:10.586727 32326 raft_consensus.cc:2454] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
W20260809 14:30:10.590838 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet d4980140ba2a4ecc87001e2a43c8e813 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4980140ba2a4ecc87001e2a43c8e813 already in progress: creating tablet
W20260809 14:30:10.591603 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 6fa25dbd78f64e5db9e31d433e8f4736 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6fa25dbd78f64e5db9e31d433e8f4736 already in progress: creating tablet
W20260809 14:30:10.592267 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 3619d49d0b944a91bcb181e884047b38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3619d49d0b944a91bcb181e884047b38 already in progress: creating tablet
W20260809 14:30:10.592938 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 301b53eeb3be41a2b8ff2598d1b531d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 301b53eeb3be41a2b8ff2598d1b531d7 already in progress: creating tablet
W20260809 14:30:10.593837 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 3d7cbb2bb26c4bd7a089648007d7f041 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d7cbb2bb26c4bd7a089648007d7f041 already in progress: creating tablet
W20260809 14:30:10.594470 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 07ac3ef5573c40b894c9b2e743ca0582 because the tablet was not found. No further retry: Not found: Tablet not found: 07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:10.595311 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5121e9f5a85a45d285d78f41a442769a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:10.598651 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29" } modify_external_catalogs: true
I20260809 14:30:10.599200 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29" } modify_external_catalogs: true
I20260809 14:30:10.605062 32705 tablet_bootstrap.cc:492] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:10.605458 32705 ts_tablet_manager.cc:1397] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.009s
I20260809 14:30:10.614059 32705 raft_consensus.cc:348] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.614696 32705 raft_consensus.cc:374] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.614939 32705 raft_consensus.cc:729] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.616848 32693 consensus_queue.cc:260] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.624306 732 consensus_queue.cc:1035] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260809 14:30:10.623335 32705 consensus_queue.cc:260] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.630105 32705 ts_tablet_manager.cc:1428] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.024s user 0.004s sys 0.001s
I20260809 14:30:10.630963 32705 tablet_bootstrap.cc:492] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:10.632645 32443 leader_election.cc:341] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.633186 32443 leader_election.cc:302] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:10.634591 32443 leader_election.cc:302] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:10.635684 32443 leader_election.cc:341] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.635990 32443 leader_election.cc:302] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:10.638062 31938 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3228a1bca97b4dfbbbcb8095a1da792e" } modify_external_catalogs: true
I20260809 14:30:10.638571 31938 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3228a1bca97b4dfbbbcb8095a1da792e" } modify_external_catalogs: true
I20260809 14:30:10.643183 730 raft_consensus.cc:2726] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:10.643823 673 raft_consensus.cc:2781] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:10.644168 673 raft_consensus.cc:482] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:10.644491 673 raft_consensus.cc:3037] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:10.649061 732 raft_consensus.cc:2726] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:10.651497 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:10.652155 32515 raft_consensus.cc:2454] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:10.654585 32479 tablet_service.cc:1467] Processing CreateTablet for tablet f43385579b29417289f3bdacf8fce278 (DEFAULT_TABLE table=default.Table_bea4b6049d234a58a4bc8222fd20f462 [id=2a97745346124303a9dce29dbef6727c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.655839 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f43385579b29417289f3bdacf8fce278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.661700 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 9c890a8c73044f8ba433151c4efe17af (DEFAULT_TABLE table=default.Table_22a408c4e94c43a29e31198ad59a5e1b [id=ff2c2c4e4811464b97e4bf61226ca6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.662736 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c890a8c73044f8ba433151c4efe17af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.670378 32705 tablet_bootstrap.cc:654] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.670943 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:10.671624 32515 raft_consensus.cc:3037] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:10.678575 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet b0ad3946f5de4387af3627a88b6404cf (table default.Table_bb02de675fd247e5af1ff74a6c57f2bd [id=eb07bcad144c4719ae97a233970493f8]) successfully deleted
W20260809 14:30:10.679194 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 273ca6176a184a719538a9455afded4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 273ca6176a184a719538a9455afded4d already in progress: creating tablet
I20260809 14:30:10.682600 32693 ts_tablet_manager.cc:1428] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.157s user 0.005s sys 0.000s
W20260809 14:30:10.682931 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 37a3a71874fd44bf865b11c05e84ca84 consensus state has no leader
I20260809 14:30:10.685284 32693 tablet_bootstrap.cc:492] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:10.693025 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5ae34de65afc4d89bd9c6dc099018263"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:10.694685 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ae34de65afc4d89bd9c6dc099018263 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:10.701119 32478 tablet_service.cc:1514] Processing DeleteTablet for tablet eba8c216630d482db5bf235b4c8f5b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:09 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:10.702502 32478 tablet_service.cc:1514] Processing DeleteTablet for tablet ac27f377aa824654b6491c3065ef819c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:09 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:10.703588 32478 tablet_service.cc:1514] Processing DeleteTablet for tablet d737c419ce924d2d982474c74019c472 with delete_type TABLET_DATA_DELETED (replaced by tablet ed328e6f95544c25b08075018cb8caf2) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:10.705246 32478 tablet_service.cc:1467] Processing CreateTablet for tablet e0acb71107a74d178b3512df1ca2f140 (DEFAULT_TABLE table=default.Table_fb1e9afadc124d26b601aeaaaf0d14b3 [id=ca824009fd984674bad52b95f8ba3617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.706409 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0acb71107a74d178b3512df1ca2f140. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:10.717463 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f06cf519a49441ae885a73e295fb585d consensus state has no leader
I20260809 14:30:10.720387 32691 tablet_bootstrap.cc:492] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:10.726640 32691 tablet_bootstrap.cc:654] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.731544 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 1d52f4b7d49544109df36c085e4aaec3 (DEFAULT_TABLE table=default.Table_250ec22f736f4ef885234ffff62f2054 [id=1ea46aca8f884ea28bebea0f1c9006a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:10.732810 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d52f4b7d49544109df36c085e4aaec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.734400 31939 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:10.747164 730 raft_consensus.cc:482] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:30:10.749961 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet eba8c216630d482db5bf235b4c8f5b15 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eba8c216630d482db5bf235b4c8f5b15 already in progress: creating tablet
W20260809 14:30:10.750618 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet ac27f377aa824654b6491c3065ef819c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac27f377aa824654b6491c3065ef819c already in progress: creating tablet
W20260809 14:30:10.751161 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet d737c419ce924d2d982474c74019c472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d737c419ce924d2d982474c74019c472 already in progress: creating tablet
I20260809 14:30:10.752208 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet f06cf519a49441ae885a73e295fb585d
I20260809 14:30:10.749152 730 raft_consensus.cc:504] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.753489 31940 catalog_manager.cc:6565] ReplaceTablet: tablet f06cf519a49441ae885a73e295fb585d deleted and replaced by tablet bf2fa268687d429f9fa23b182c427d6c
I20260809 14:30:10.754818 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:10.756223 730 leader_election.cc:288] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:10.756920 32439 leader_election.cc:341] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.759939 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet f06cf519a49441ae885a73e295fb585d with delete_type TABLET_DATA_DELETED (replaced by tablet bf2fa268687d429f9fa23b182c427d6c) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:10.761487 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet f06cf519a49441ae885a73e295fb585d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f06cf519a49441ae885a73e295fb585d already in progress: creating tablet
I20260809 14:30:10.762840 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet f06cf519a49441ae885a73e295fb585d with delete_type TABLET_DATA_DELETED (replaced by tablet bf2fa268687d429f9fa23b182c427d6c) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:10.769709 31938 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 5be5cc6d7cb042d1b577cb7b24a95827
I20260809 14:30:10.773403 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be5cc6d7cb042d1b577cb7b24a95827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:10.773378 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:10.745712 32477 tablet_service.cc:1467] Processing CreateTablet for tablet 66e42e58096c447db3315436c361db0b (DEFAULT_TABLE table=default.Table_76064ebb420d48dfb1586d58bf81d6a3 [id=9a90fee262334c4881f9202da51858ce]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:10.775074 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 5be5cc6d7cb042d1b577cb7b24a95827 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5be5cc6d7cb042d1b577cb7b24a95827 already in progress: creating tablet
I20260809 14:30:10.775904 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e42e58096c447db3315436c361db0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:10.782012 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be5cc6d7cb042d1b577cb7b24a95827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:10.784327 32443 leader_election.cc:341] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.784828 32443 leader_election.cc:302] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:10.787174 731 raft_consensus.cc:482] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:10.787644 731 raft_consensus.cc:504] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.789595 731 leader_election.cc:288] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:10.790314 32481 tablet_service.cc:1514] Processing DeleteTablet for tablet f06cf519a49441ae885a73e295fb585d with delete_type TABLET_DATA_DELETED (replaced by tablet bf2fa268687d429f9fa23b182c427d6c) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:10.791779 673 raft_consensus.cc:504] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.793031 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet cb799e344ae544f9b4df659462bddba2
I20260809 14:30:10.793764 673 leader_election.cc:288] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:10.794895 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_ca71a97e95374666b5d79f35075e26b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:10.796003 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:10.796355 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca71a97e95374666b5d79f35075e26b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:10.796401 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6ebda27b44048b5734e8798a980c2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:10.797716 32439 leader_election.cc:341] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.797887 685 raft_consensus.cc:482] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:30:10.798442 32439 leader_election.cc:341] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.798318 685 raft_consensus.cc:504] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.801625 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:10.802953 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b983b837948f4b0c816c275d04af0c02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:10.800216 685 leader_election.cc:288] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:10.806809 31944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 44) took 917 ms (client timeout 1000 ms). Trace:
I20260809 14:30:10.807135 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet cb799e344ae544f9b4df659462bddba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:10.804392 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b983b837948f4b0c816c275d04af0c02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:10.807156 589 consensus_queue.cc:1035] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:10.807835 353 tablet_replica.cc:330] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:10.808504 353 raft_consensus.cc:2229] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:10.808943 353 raft_consensus.cc:2258] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:10.810722 353 ts_tablet_manager.cc:1905] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:10.821405 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be5cc6d7cb042d1b577cb7b24a95827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:10.807401 31944 rpcz_store.cc:269] 0809 14:30:09.889305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:09.904565 (+ 15260us) service_pool.cc:227] Handling call
0809 14:30:10.044493 (+139928us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:10.806756 (+762263us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:10.637642 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:10.638005 (+ 363us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:10.638027 (+ 22us) write_op.cc:435] Acquired schema lock
0809 14:30:10.638037 (+ 10us) tablet.cc:660] Decoding operations
0809 14:30:10.638293 (+ 256us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:10.638325 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:10.638346 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:10.638481 (+ 135us) tablet.cc:699] Row locks acquired
0809 14:30:10.638495 (+ 14us) write_op.cc:260] PREPARE: finished
0809 14:30:10.638600 (+ 105us) write_op.cc:270] Start()
0809 14:30:10.638690 (+ 90us) write_op.cc:276] Timestamp: P: 1786285810638539 usec, L: 0
0809 14:30:10.638705 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 14:30:10.639166 (+ 461us) log.cc:842] Serialized 533 byte log entry
0809 14:30:10.706117 (+ 66951us) op_driver.cc:444] REPLICATION: finished
0809 14:30:10.711844 (+ 5727us) write_op.cc:301] APPLY: starting
0809 14:30:10.711895 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:10.711978 (+ 83us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:10.712085 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:10.712236 (+ 151us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:10.712362 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:10.712406 (+ 44us) write_op.cc:312] APPLY: finished
0809 14:30:10.729583 (+ 17177us) log.cc:842] Serialized 23 byte log entry
0809 14:30:10.729739 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:10.729807 (+ 68us) write_op.cc:454] Released schema lock
0809 14:30:10.729842 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:10.731521 (+ 1679us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:10.731548 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":301672704,"child_traces":[["op",{"apply.queue_time_us":5613,"apply.run_cpu_time_us":1442,"apply.run_wall_time_us":19857,"num_ops":1,"prepare.queue_time_us":396415,"prepare.run_cpu_time_us":1723,"prepare.run_wall_time_us":1723,"replication_time_us":67332}]]}
I20260809 14:30:10.823150 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:10.824839 32095 leader_election.cc:302] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:10.825781 685 raft_consensus.cc:2726] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:10.829838 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 45550.1us
W20260809 14:30:10.831863 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet f06cf519a49441ae885a73e295fb585d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f06cf519a49441ae885a73e295fb585d already in progress: creating tablet
I20260809 14:30:10.832525 673 raft_consensus.cc:2726] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:10.832671 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 5be5cc6d7cb042d1b577cb7b24a95827 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5be5cc6d7cb042d1b577cb7b24a95827 already in progress: creating tablet
I20260809 14:30:10.834339 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet cb799e344ae544f9b4df659462bddba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:10.835021 626 tablet_replica.cc:330] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:10.835811 626 raft_consensus.cc:2229] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:10.836851 626 raft_consensus.cc:2258] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:10.845206 626 ts_tablet_manager.cc:1905] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:10.853832 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet f06cf519a49441ae885a73e295fb585d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f06cf519a49441ae885a73e295fb585d already in progress: creating tablet
W20260809 14:30:10.854583 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 5be5cc6d7cb042d1b577cb7b24a95827 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5be5cc6d7cb042d1b577cb7b24a95827 already in progress: creating tablet
I20260809 14:30:10.865448 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 57ee617d858c42f7b03bd287a33eebe4 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:10.871891 353 ts_tablet_manager.cc:1918] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:10.875088 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet cb799e344ae544f9b4df659462bddba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:10.876331 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad3946f5de4387af3627a88b6404cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:10.881450 426 tablet_replica.cc:330] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:10.882293 426 raft_consensus.cc:2229] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:10.882823 426 raft_consensus.cc:2258] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:10.886894 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_647afce4319c40c7ac1de82ead70b0c4" } new_table_name: "default.Table_32fd1fbe06e041adbd3b9aa6023f4972" modify_external_catalogs: true
I20260809 14:30:10.890050 32705 tablet_bootstrap.cc:492] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:10.890522 32705 ts_tablet_manager.cc:1397] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.260s user 0.011s sys 0.011s
I20260809 14:30:10.890661 32691 tablet_bootstrap.cc:492] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:10.891083 32691 ts_tablet_manager.cc:1397] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.171s user 0.012s sys 0.000s
I20260809 14:30:10.893479 426 ts_tablet_manager.cc:1905] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:10.900527 32705 raft_consensus.cc:348] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.882728 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6ebda27b44048b5734e8798a980c2" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:10.901298 32705 raft_consensus.cc:374] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:10.901548 32705 raft_consensus.cc:729] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:10.901635 32326 raft_consensus.cc:3037] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:10.906469 32705 consensus_queue.cc:260] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:10.911481 32326 raft_consensus.cc:2454] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
W20260809 14:30:10.915201 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 95733a086a704e1ea397d4ffb2f2c5e7 has no consensus state
I20260809 14:30:10.917194 32515 raft_consensus.cc:2454] T ae1af25739774df8a5536c75c2e1f416 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:10.919159 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_ed4114d732c44266bc8e100a01ce3499"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:10.920717 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed4114d732c44266bc8e100a01ce3499 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:10.926493 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5ae34de65afc4d89bd9c6dc099018263" } new_table_name: "default.Table_de805f1551304f6a92cf7d4db934dcda" modify_external_catalogs: true
I20260809 14:30:10.933672 32705 ts_tablet_manager.cc:1428] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.043s user 0.008s sys 0.000s
I20260809 14:30:10.934569 32705 tablet_bootstrap.cc:492] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:10.912303 353 log.cc:1196] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/cb799e344ae544f9b4df659462bddba2
I20260809 14:30:10.937740 353 ts_tablet_manager.cc:1939] T cb799e344ae544f9b4df659462bddba2 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:10.940378 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]) successfully deleted
I20260809 14:30:10.903401 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:10.958950 32443 leader_election.cc:302] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:10.960501 32443 leader_election.cc:341] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:10.961629 32443 leader_election.cc:302] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:10.961140 732 raft_consensus.cc:2781] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:10.964704 673 raft_consensus.cc:2726] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:10.972949 626 ts_tablet_manager.cc:1918] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:10.973665 626 log.cc:1196] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/cb799e344ae544f9b4df659462bddba2
I20260809 14:30:10.974444 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_05c045fe502848f4805ad20c9362876f" } modify_external_catalogs: true
I20260809 14:30:10.975005 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_05c045fe502848f4805ad20c9362876f" } modify_external_catalogs: true
I20260809 14:30:10.976626 732 raft_consensus.cc:686] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:10.976897 626 ts_tablet_manager.cc:1939] T cb799e344ae544f9b4df659462bddba2 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:10.980643 32705 tablet_bootstrap.cc:654] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:10.983036 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_6cb89cdda08047ada14df086bf662194"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:10.977847 732 consensus_queue.cc:237] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:10.994912 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cb89cdda08047ada14df086bf662194 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:10.997226 32691 raft_consensus.cc:348] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.001827 32169 raft_consensus.cc:3037] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.004281 31948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e329e3111b054b7db1bdeb7e4a91720d" } modify_external_catalogs: true
I20260809 14:30:11.004856 31948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e329e3111b054b7db1bdeb7e4a91720d" } modify_external_catalogs: true
I20260809 14:30:10.997890 32691 raft_consensus.cc:374] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:11.012306 32691 raft_consensus.cc:729] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:11.009557 32693 tablet_bootstrap.cc:654] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:11.013891 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]) successfully deleted
I20260809 14:30:11.028622 754 raft_consensus.cc:482] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.029399 754 raft_consensus.cc:504] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.034348 754 leader_election.cc:288] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:11.034910 32705 tablet_bootstrap.cc:492] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:11.035377 32705 ts_tablet_manager.cc:1397] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.101s user 0.009s sys 0.005s
I20260809 14:30:11.035189 754 consensus_queue.cc:1035] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:11.039783 32705 raft_consensus.cc:348] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.040814 32691 consensus_queue.cc:260] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:11.042667 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9121614514574d0594b1d5fe89fcc366 has no consensus state
I20260809 14:30:11.047477 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_657bee986ba04a85aa02aff63c665910"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:11.049202 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_657bee986ba04a85aa02aff63c665910 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:11.049718 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:11.055621 32515 raft_consensus.cc:2454] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:11.057843 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_d34159d14bb64162bda06db2b81cf6c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:11.059788 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d34159d14bb64162bda06db2b81cf6c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:11.061621 32260 leader_election.cc:302] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:11.040510 32705 raft_consensus.cc:374] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:11.062996 32705 raft_consensus.cc:729] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:11.068674 754 raft_consensus.cc:482] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.068385 589 raft_consensus.cc:2781] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:11.070261 589 raft_consensus.cc:482] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:11.069864 754 raft_consensus.cc:504] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.072471 589 raft_consensus.cc:3037] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.034811 426 ts_tablet_manager.cc:1918] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:11.075707 426 log.cc:1196] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/cb799e344ae544f9b4df659462bddba2
I20260809 14:30:11.077222 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:11.078976 426 ts_tablet_manager.cc:1939] T cb799e344ae544f9b4df659462bddba2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:11.063705 32705 consensus_queue.cc:260] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.083729 32693 tablet_bootstrap.cc:492] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
W20260809 14:30:11.085129 32260 leader_election.cc:341] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.089175 32693 ts_tablet_manager.cc:1397] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.405s user 0.017s sys 0.005s
I20260809 14:30:11.097116 673 raft_consensus.cc:482] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.097626 673 raft_consensus.cc:504] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.103883 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:11.105647 32443 leader_election.cc:341] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.106261 673 leader_election.cc:288] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:11.110329 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:11.110976 32705 ts_tablet_manager.cc:1428] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.075s user 0.005s sys 0.005s
I20260809 14:30:11.111204 32168 raft_consensus.cc:2454] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:30:11.112923 732 raft_consensus.cc:482] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.113509 32439 leader_election.cc:302] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:30:11.115031 32691 ts_tablet_manager.cc:1428] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.224s user 0.010s sys 0.000s
I20260809 14:30:11.118765 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:11.119453 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 2a689385d063469595fcae82aa979bb1 (table default.Table_d841d100269948ca8f7fc467c85a3b0d [id=c19378f8e8994ef28e7b0eba079948c3]). Expected version 2 got 0
I20260809 14:30:11.120088 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:11.113494 732 raft_consensus.cc:504] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.122527 31939 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:11.122443 31937 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b9541d0208bb4c938aff912a0df84a22" } new_table_name: "default.Table_42a687657ecc485a9c3d956e245784c5" modify_external_catalogs: true
I20260809 14:30:11.123165 31939 catalog_manager.cc:5495] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65). New cstate: current_term: 1 leader_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:11.115586 673 raft_consensus.cc:2781] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:11.124306 673 raft_consensus.cc:482] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:11.124874 673 raft_consensus.cc:3037] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.128237 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet cb799e344ae544f9b4df659462bddba2 (table default.Table_7a42eb872b9d4fbe93c6f4fe5fc22f29 [id=6866e71fe868487cb904cab993a14371]) successfully deleted
I20260809 14:30:11.130158 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 46d8d00bbf2e4f4289448e48536979ea (DEFAULT_TABLE table=default.Table_478fce1bf3d54075835a5d464370d13e [id=124038878ed44174a413bd6ffa2e836d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.130825 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c4cbdda4ed48a8b34728d231497d02" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:11.131927 754 leader_election.cc:288] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:11.136305 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ee617d858c42f7b03bd287a33eebe4" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:11.137018 32515 raft_consensus.cc:2454] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:11.137207 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d8d00bbf2e4f4289448e48536979ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.138641 32260 leader_election.cc:302] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:11.139758 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c4cbdda4ed48a8b34728d231497d02" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:11.141332 32443 leader_election.cc:341] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.142660 32705 tablet_bootstrap.cc:492] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:11.144584 32439 leader_election.cc:341] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.144716 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ee617d858c42f7b03bd287a33eebe4" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:11.144977 32439 leader_election.cc:302] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:11.145689 730 raft_consensus.cc:2726] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:11.145959 754 raft_consensus.cc:2781] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:11.146571 754 raft_consensus.cc:482] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:11.147272 754 raft_consensus.cc:3037] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.148622 31939 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
W20260809 14:30:11.150907 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet b0ad3946f5de4387af3627a88b6404cf because the tablet was not found. No further retry: Not found: Tablet not found: b0ad3946f5de4387af3627a88b6404cf
W20260809 14:30:11.155117 32260 leader_election.cc:341] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.156294 732 leader_election.cc:288] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:11.161031 685 raft_consensus.cc:482] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.161739 685 raft_consensus.cc:504] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.162314 754 raft_consensus.cc:504] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.166152 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5121e9f5a85a45d285d78f41a442769a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:11.167690 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ee617d858c42f7b03bd287a33eebe4" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:11.167717 685 leader_election.cc:288] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:11.170409 32260 leader_election.cc:341] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.170975 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5121e9f5a85a45d285d78f41a442769a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:11.171890 32705 tablet_bootstrap.cc:654] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:11.171814 32515 raft_consensus.cc:2454] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:11.172165 754 leader_election.cc:288] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:11.173273 32095 leader_election.cc:302] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:11.173414 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ee617d858c42f7b03bd287a33eebe4" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:11.174257 32512 raft_consensus.cc:3037] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.174410 685 raft_consensus.cc:2781] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:11.175218 685 raft_consensus.cc:482] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:11.175803 685 raft_consensus.cc:3037] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.191496 685 raft_consensus.cc:504] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.193746 32691 tablet_bootstrap.cc:492] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:11.198635 761 tablet.cc:1722] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:11.204772 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_6cabfb77009541cabeb67c8fd017826e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:11.206470 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cabfb77009541cabeb67c8fd017826e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:11.208928 762 tablet.cc:1722] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:11.215236 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 930fb8d5f682462183c780ab52ce6dd6 (DEFAULT_TABLE table=default.Table_bc907356c8584717b336afc1a260018e [id=ae8d88f6b2ff44a49e4eb4bf10fe996c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.216394 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930fb8d5f682462183c780ab52ce6dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.217576 685 leader_election.cc:288] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:11.217900 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5121e9f5a85a45d285d78f41a442769a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:11.233342 32515 raft_consensus.cc:3037] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.235800 32478 tablet_service.cc:1467] Processing CreateTablet for tablet 930fb8d5f682462183c780ab52ce6dd6 (DEFAULT_TABLE table=default.Table_bc907356c8584717b336afc1a260018e [id=ae8d88f6b2ff44a49e4eb4bf10fe996c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.236145 32512 raft_consensus.cc:2454] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:11.236922 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930fb8d5f682462183c780ab52ce6dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.241700 32515 raft_consensus.cc:2454] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:11.243124 32095 leader_election.cc:302] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
W20260809 14:30:11.232832 758 log.cc:924] Time spent T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260809 14:30:11.252146 32691 tablet_bootstrap.cc:654] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:11.254743 685 raft_consensus.cc:2781] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:11.255393 685 raft_consensus.cc:686] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 LEADER]: Becoming Leader. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Running, Role: LEADER
I20260809 14:30:11.256196 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 71ce43b8a46c4344bf444cd24c3294f7 (DEFAULT_TABLE table=default.Table_feb15cc2cdf64b3bb4c6de7992d27e89 [id=97e27a6912094a968b01f0acaded2266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.256139 685 consensus_queue.cc:237] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.211601 32298 tablet_service.cc:1467] Processing CreateTablet for tablet 46d8d00bbf2e4f4289448e48536979ea (DEFAULT_TABLE table=default.Table_478fce1bf3d54075835a5d464370d13e [id=124038878ed44174a413bd6ffa2e836d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.257411 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ce43b8a46c4344bf444cd24c3294f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.258453 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d8d00bbf2e4f4289448e48536979ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.215334 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5121e9f5a85a45d285d78f41a442769a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:11.233090 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 930fb8d5f682462183c780ab52ce6dd6 (DEFAULT_TABLE table=default.Table_bc907356c8584717b336afc1a260018e [id=ae8d88f6b2ff44a49e4eb4bf10fe996c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.267979 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930fb8d5f682462183c780ab52ce6dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.269209 673 raft_consensus.cc:504] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.271809 673 leader_election.cc:288] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:11.237502 32260 leader_election.cc:302] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:11.247244 589 raft_consensus.cc:504] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.273739 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:11.274274 589 leader_election.cc:288] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:11.274374 32167 raft_consensus.cc:3037] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.274729 589 raft_consensus.cc:2781] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:11.275138 589 raft_consensus.cc:686] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:11.121366 32693 raft_consensus.cc:348] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.277494 32693 raft_consensus.cc:374] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:11.277741 32693 raft_consensus.cc:729] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:11.278296 32693 consensus_queue.cc:260] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.281318 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:11.275864 589 consensus_queue.cc:237] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:11.283174 32443 leader_election.cc:341] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.285249 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:11.285533 32167 raft_consensus.cc:2454] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:11.285910 32512 raft_consensus.cc:3037] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.285580 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:11.286530 32439 leader_election.cc:302] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b
I20260809 14:30:11.287425 673 raft_consensus.cc:2781] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:11.287858 673 raft_consensus.cc:686] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:11.288762 32693 ts_tablet_manager.cc:1428] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.199s user 0.003s sys 0.004s
I20260809 14:30:11.288733 673 consensus_queue.cc:237] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.289711 32693 tablet_bootstrap.cc:492] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
W20260809 14:30:11.293058 32260 leader_election.cc:341] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.295182 32329 raft_consensus.cc:2454] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 0.
I20260809 14:30:11.296029 32691 tablet_bootstrap.cc:492] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:11.296561 32691 ts_tablet_manager.cc:1397] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.103s user 0.013s sys 0.000s
I20260809 14:30:11.297050 32512 raft_consensus.cc:2454] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:11.298172 32260 leader_election.cc:302] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:11.298882 754 raft_consensus.cc:2781] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:11.298900 32691 raft_consensus.cc:348] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.299290 754 raft_consensus.cc:686] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:11.299541 32691 raft_consensus.cc:374] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:11.299772 32691 raft_consensus.cc:729] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:11.299997 754 consensus_queue.cc:237] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.302366 32691 consensus_queue.cc:260] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.312232 32691 ts_tablet_manager.cc:1428] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260809 14:30:11.314116 32304 tablet_service.cc:1467] Processing CreateTablet for tablet 71ce43b8a46c4344bf444cd24c3294f7 (DEFAULT_TABLE table=default.Table_feb15cc2cdf64b3bb4c6de7992d27e89 [id=97e27a6912094a968b01f0acaded2266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.315886 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ce43b8a46c4344bf444cd24c3294f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.319264 32691 tablet_bootstrap.cc:492] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:11.321812 32693 tablet_bootstrap.cc:654] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:11.321813 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 71ce43b8a46c4344bf444cd24c3294f7 (DEFAULT_TABLE table=default.Table_feb15cc2cdf64b3bb4c6de7992d27e89 [id=97e27a6912094a968b01f0acaded2266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.323328 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ce43b8a46c4344bf444cd24c3294f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.328076 32691 tablet_bootstrap.cc:654] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:11.305809 32477 tablet_service.cc:1467] Processing CreateTablet for tablet 46d8d00bbf2e4f4289448e48536979ea (DEFAULT_TABLE table=default.Table_478fce1bf3d54075835a5d464370d13e [id=124038878ed44174a413bd6ffa2e836d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.350962 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f4fdd6821b4749b4869ebd018c1e7ed7" } new_table_name: "default.Table_d88adbc3d1d2475eab5db97ea2f6260a" modify_external_catalogs: true
I20260809 14:30:11.369643 32136 tablet_service.cc:1467] Processing CreateTablet for tablet 8beca5ba337b4130aadcb0a80f38750e (DEFAULT_TABLE table=default.Table_385287d0f4f64e54a624637cdb94d2f9 [id=938a06aa7567427eac5789dc6735ccb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.371564 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8beca5ba337b4130aadcb0a80f38750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.374974 685 raft_consensus.cc:482] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.375749 685 raft_consensus.cc:504] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.377494 32298 tablet_service.cc:1467] Processing CreateTablet for tablet 8beca5ba337b4130aadcb0a80f38750e (DEFAULT_TABLE table=default.Table_385287d0f4f64e54a624637cdb94d2f9 [id=938a06aa7567427eac5789dc6735ccb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.379083 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8beca5ba337b4130aadcb0a80f38750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.380749 685 leader_election.cc:288] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:11.382640 685 raft_consensus.cc:482] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.383525 685 raft_consensus.cc:504] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.388801 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6ebda27b44048b5734e8798a980c2" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:11.390727 32329 raft_consensus.cc:2379] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
W20260809 14:30:11.393754 32099 leader_election.cc:341] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:11.395820 685 leader_election.cc:288] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:11.398518 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d8d00bbf2e4f4289448e48536979ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:11.400771 32095 leader_election.cc:341] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:11.401193 32095 leader_election.cc:302] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:11.402818 735 raft_consensus.cc:2726] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:11.411360 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6ebda27b44048b5734e8798a980c2" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:11.411933 32141 tablet_service.cc:1467] Processing CreateTablet for tablet 95733a086a704e1ea397d4ffb2f2c5e7 (DEFAULT_TABLE table=default.Table_32fd1fbe06e041adbd3b9aa6023f4972 [id=aecc31c077094c58bc27e1ffdf0ea878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.413738 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95733a086a704e1ea397d4ffb2f2c5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.417497 32479 tablet_service.cc:1467] Processing CreateTablet for tablet 8beca5ba337b4130aadcb0a80f38750e (DEFAULT_TABLE table=default.Table_385287d0f4f64e54a624637cdb94d2f9 [id=938a06aa7567427eac5789dc6735ccb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.419203 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8beca5ba337b4130aadcb0a80f38750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.419449 32095 leader_election.cc:302] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:11.427367 735 raft_consensus.cc:3037] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.466332 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_054e300a0545424c8148c2285469944e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:11.468117 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_054e300a0545424c8148c2285469944e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:11.493263 32691 tablet_bootstrap.cc:492] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:11.493796 32691 ts_tablet_manager.cc:1397] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.175s user 0.019s sys 0.017s
I20260809 14:30:11.495676 685 consensus_queue.cc:1035] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:11.499953 735 raft_consensus.cc:2726] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:11.444842 673 raft_consensus.cc:482] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.504508 673 raft_consensus.cc:504] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.507609 673 leader_election.cc:288] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:11.521193 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 95733a086a704e1ea397d4ffb2f2c5e7 (DEFAULT_TABLE table=default.Table_32fd1fbe06e041adbd3b9aa6023f4972 [id=aecc31c077094c58bc27e1ffdf0ea878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.528440 32691 raft_consensus.cc:348] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.544554 32691 raft_consensus.cc:374] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:11.544834 32691 raft_consensus.cc:729] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:11.546562 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:11.471194 31937 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a0b196e429514cbc803e9a99a986d4d2 consensus state has no leader
I20260809 14:30:11.547644 32691 consensus_queue.cc:260] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.548183 31937 master_service.cc:946] ReplaceTablet: received request to replace tablet 29cd0861b04747699ab2d13cb89b418d from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:11.476864 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 7856076c1332479a894d2d69d16b19f5
I20260809 14:30:11.548959 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95733a086a704e1ea397d4ffb2f2c5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.556686 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet 7856076c1332479a894d2d69d16b19f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:11.560132 32443 leader_election.cc:341] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.549278 754 raft_consensus.cc:482] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.561558 754 raft_consensus.cc:504] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.567651 32691 ts_tablet_manager.cc:1428] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.074s user 0.009s sys 0.000s
I20260809 14:30:11.568813 32691 tablet_bootstrap.cc:492] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:30:11.574843 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3d81046d929c48a3a6946f20d8fd7bca (table default.Table_4b848656d0ca4f2b81bd9e56d4540967 [id=f60105c3fd98462b9367f400abca2c15], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3d81046d929c48a3a6946f20d8fd7bca consensus state has no leader
I20260809 14:30:11.570230 32478 tablet_service.cc:1514] Processing DeleteTablet for tablet 7856076c1332479a894d2d69d16b19f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:11.579553 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 7856076c1332479a894d2d69d16b19f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7856076c1332479a894d2d69d16b19f5 already in progress: creating tablet
I20260809 14:30:11.508946 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:11.558738 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 7856076c1332479a894d2d69d16b19f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:10 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:11.566213 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:11.567422 780 consensus_queue.cc:1035] T ae1af25739774df8a5536c75c2e1f416 P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:11.584436 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 946447e981224c818b5d8e0079d60559 (DEFAULT_TABLE table=default.Table_ed937fcc529a4bad9edc6085fdf5b47d [id=a783e9da7fa5498eaaf21e95b8c5a292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.587981 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 946447e981224c818b5d8e0079d60559. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:11.588717 32260 leader_election.cc:341] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:11.590665 32439 leader_election.cc:341] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.591096 32439 leader_election.cc:302] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:11.594017 673 raft_consensus.cc:2726] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:11.601508 32478 tablet_service.cc:1467] Processing CreateTablet for tablet 95733a086a704e1ea397d4ffb2f2c5e7 (DEFAULT_TABLE table=default.Table_32fd1fbe06e041adbd3b9aa6023f4972 [id=aecc31c077094c58bc27e1ffdf0ea878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.608896 754 leader_election.cc:288] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:11.621977 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95733a086a704e1ea397d4ffb2f2c5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.622741 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:11.600288 32691 tablet_bootstrap.cc:654] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:11.623420 32515 raft_consensus.cc:2454] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
W20260809 14:30:11.626752 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 7856076c1332479a894d2d69d16b19f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7856076c1332479a894d2d69d16b19f5 already in progress: creating tablet
I20260809 14:30:11.637502 32705 tablet_bootstrap.cc:492] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:11.637948 32705 ts_tablet_manager.cc:1397] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.495s user 0.008s sys 0.007s
I20260809 14:30:11.641321 32691 tablet_bootstrap.cc:492] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:11.641708 32691 ts_tablet_manager.cc:1397] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.073s user 0.007s sys 0.007s
I20260809 14:30:11.653420 32260 leader_election.cc:302] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:11.654549 589 raft_consensus.cc:2781] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:11.654917 589 raft_consensus.cc:482] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:11.655261 589 raft_consensus.cc:3037] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.690644 32691 raft_consensus.cc:348] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.691552 32691 raft_consensus.cc:374] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:11.691912 32691 raft_consensus.cc:729] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:11.706897 32691 consensus_queue.cc:260] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.709811 32691 ts_tablet_manager.cc:1428] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.068s user 0.005s sys 0.003s
I20260809 14:30:11.710774 32691 tablet_bootstrap.cc:492] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:11.734684 32705 raft_consensus.cc:348] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.735515 32705 raft_consensus.cc:374] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:11.735834 32705 raft_consensus.cc:729] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:11.741849 32705 consensus_queue.cc:260] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.745517 32148 tablet_service.cc:1467] Processing CreateTablet for tablet 5b802030779d423082b676f99b617a1e (DEFAULT_TABLE table=default.Table_09716b0be3174d30bc3adf0383d40efd [id=4ec3081646e84f14ba0605bbb51e2f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.746793 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b802030779d423082b676f99b617a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.750604 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 5b802030779d423082b676f99b617a1e (DEFAULT_TABLE table=default.Table_09716b0be3174d30bc3adf0383d40efd [id=4ec3081646e84f14ba0605bbb51e2f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.751626 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 3d3bbd5a454947e29d34620c5073fb90 (DEFAULT_TABLE table=default.Table_a7b9b0894c4a47afb6848436134e53fc [id=20f5ef70cc1845c9ba5e3a6d3b1a9c1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.752336 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b802030779d423082b676f99b617a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.757166 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d3bbd5a454947e29d34620c5073fb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.762097 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 3d3bbd5a454947e29d34620c5073fb90 (DEFAULT_TABLE table=default.Table_a7b9b0894c4a47afb6848436134e53fc [id=20f5ef70cc1845c9ba5e3a6d3b1a9c1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.761809 589 raft_consensus.cc:504] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.763207 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d3bbd5a454947e29d34620c5073fb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.767629 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 5b802030779d423082b676f99b617a1e (DEFAULT_TABLE table=default.Table_09716b0be3174d30bc3adf0383d40efd [id=4ec3081646e84f14ba0605bbb51e2f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.768935 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b802030779d423082b676f99b617a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.771900 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 3d3bbd5a454947e29d34620c5073fb90 (DEFAULT_TABLE table=default.Table_a7b9b0894c4a47afb6848436134e53fc [id=20f5ef70cc1845c9ba5e3a6d3b1a9c1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.773303 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d3bbd5a454947e29d34620c5073fb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.774070 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 8c06535e3b5e441c89e9db7431267f98 (DEFAULT_TABLE table=default.Table_43f73cfcef6546fb985efbe413dff035 [id=72bef9dd3c5e43dc9e77990689fb7732]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.775286 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c06535e3b5e441c89e9db7431267f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.776355 32693 tablet_bootstrap.cc:492] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:11.776809 32693 ts_tablet_manager.cc:1397] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.487s user 0.003s sys 0.013s
I20260809 14:30:11.789263 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 8c06535e3b5e441c89e9db7431267f98 (DEFAULT_TABLE table=default.Table_43f73cfcef6546fb985efbe413dff035 [id=72bef9dd3c5e43dc9e77990689fb7732]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.790766 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c06535e3b5e441c89e9db7431267f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.805375 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 8c06535e3b5e441c89e9db7431267f98 (DEFAULT_TABLE table=default.Table_43f73cfcef6546fb985efbe413dff035 [id=72bef9dd3c5e43dc9e77990689fb7732]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.806427 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c06535e3b5e441c89e9db7431267f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.810125 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 0b80e4ff9e3742159be8a78c633c0f56 (DEFAULT_TABLE table=default.Table_18d2bdbeb7ad42e9b0edf59e2e89e8b3 [id=e28205160a9f4582bc7b96306becf137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.811288 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b80e4ff9e3742159be8a78c633c0f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.811549 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 0b80e4ff9e3742159be8a78c633c0f56 (DEFAULT_TABLE table=default.Table_18d2bdbeb7ad42e9b0edf59e2e89e8b3 [id=e28205160a9f4582bc7b96306becf137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.811911 32479 tablet_service.cc:1467] Processing CreateTablet for tablet 946447e981224c818b5d8e0079d60559 (DEFAULT_TABLE table=default.Table_ed937fcc529a4bad9edc6085fdf5b47d [id=a783e9da7fa5498eaaf21e95b8c5a292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.812595 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b80e4ff9e3742159be8a78c633c0f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.812829 673 raft_consensus.cc:482] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.813263 673 raft_consensus.cc:504] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.815471 673 leader_election.cc:288] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:11.819339 32481 tablet_service.cc:1467] Processing CreateTablet for tablet 0b80e4ff9e3742159be8a78c633c0f56 (DEFAULT_TABLE table=default.Table_18d2bdbeb7ad42e9b0edf59e2e89e8b3 [id=e28205160a9f4582bc7b96306becf137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.821956 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b80e4ff9e3742159be8a78c633c0f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.832845 32705 ts_tablet_manager.cc:1428] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.195s user 0.009s sys 0.000s
I20260809 14:30:11.833957 32705 tablet_bootstrap.cc:492] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:11.841780 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 946447e981224c818b5d8e0079d60559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.850862 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 47089009baac430f9bcecec5dbe87687 (DEFAULT_TABLE table=default.Table_7077d276f84a485c840fded026cca74a [id=15a8f74e5d2d458abd4779617f33f2ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.851856 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47089009baac430f9bcecec5dbe87687. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:11.853566 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 184969us
I20260809 14:30:11.854507 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_be768f7988814d00aeb5aea5c1b5c91c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:11.856055 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be768f7988814d00aeb5aea5c1b5c91c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:11.878191 32691 tablet_bootstrap.cc:654] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:11.878533 32304 consensus_meta.cc:287] T 71ce43b8a46c4344bf444cd24c3294f7 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.530s user 0.007s sys 0.005s
I20260809 14:30:11.780232 32693 raft_consensus.cc:348] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.880753 32693 raft_consensus.cc:374] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:11.881023 32693 raft_consensus.cc:729] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:11.881840 32693 consensus_queue.cc:260] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.902596 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 946447e981224c818b5d8e0079d60559 (DEFAULT_TABLE table=default.Table_ed937fcc529a4bad9edc6085fdf5b47d [id=a783e9da7fa5498eaaf21e95b8c5a292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.904372 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 946447e981224c818b5d8e0079d60559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.913764 673 raft_consensus.cc:482] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:11.914438 673 raft_consensus.cc:504] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.920982 673 leader_election.cc:288] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:11.920876 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:11.922665 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:11.923913 32443 leader_election.cc:341] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:11.925100 32443 leader_election.cc:341] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.933352 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:11.933948 32512 raft_consensus.cc:3037] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:11.936869 32693 ts_tablet_manager.cc:1428] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.160s user 0.004s sys 0.004s
I20260809 14:30:11.941000 32512 raft_consensus.cc:2454] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:11.942205 32260 leader_election.cc:302] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:11.943121 754 raft_consensus.cc:2781] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:11.943612 754 raft_consensus.cc:686] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:11.945096 32693 tablet_bootstrap.cc:492] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:11.944519 754 consensus_queue.cc:237] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:11.955278 589 leader_election.cc:288] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:11.956599 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 7856076c1332479a894d2d69d16b19f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7856076c1332479a894d2d69d16b19f5 already in progress: creating tablet
I20260809 14:30:11.958067 32705 tablet_bootstrap.cc:654] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:11.966770 32693 tablet_bootstrap.cc:654] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:11.973481 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:11.974987 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:11.978825 673 consensus_queue.cc:1035] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 14:30:11.981112 32439 leader_election.cc:341] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.981549 32439 leader_election.cc:302] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:11.982585 673 raft_consensus.cc:2726] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:11.983556 32439 leader_election.cc:341] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:11.983906 32439 leader_election.cc:302] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:11.984912 673 raft_consensus.cc:2726] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:11.986285 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:11.994205 32301 tablet_service.cc:1467] Processing CreateTablet for tablet ed328e6f95544c25b08075018cb8caf2 (DEFAULT_TABLE table=default.Table_853333f626c34bc18f5a1245314d7e99 [id=504e8c38c3f1441ebe085ccbcbdab567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.994824 32299 tablet_service.cc:1467] Processing CreateTablet for tablet d9deca4f10b548ea9aff5cfdbccc91ab (DEFAULT_TABLE table=default.Table_b342ca5d31c14a89baa7ed0ea2bf8da8 [id=6efa71f406734c998668150f80e73f12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:11.995241 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed328e6f95544c25b08075018cb8caf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:11.996115 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 24b10fe71bb44ea1ae82d0232c75daf2 (DEFAULT_TABLE table=default.Table_e5e6a1c2957d4ebfaa299a5bf66454e0 [id=98f3952dbc8a4bb0a950019a58fb1d72]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:11.996671 32260 leader_election.cc:341] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:12.002502 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b10fe71bb44ea1ae82d0232c75daf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.005062 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9deca4f10b548ea9aff5cfdbccc91ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.008270 32693 tablet_bootstrap.cc:492] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:12.008729 32693 ts_tablet_manager.cc:1397] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.064s user 0.010s sys 0.000s
I20260809 14:30:12.010958 32693 raft_consensus.cc:348] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.011611 32693 raft_consensus.cc:374] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.011900 32693 raft_consensus.cc:729] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.018616 32691 tablet_bootstrap.cc:492] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:12.019197 32691 ts_tablet_manager.cc:1397] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.309s user 0.012s sys 0.002s
I20260809 14:30:12.019865 32693 consensus_queue.cc:260] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.025080 32693 ts_tablet_manager.cc:1428] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260809 14:30:12.025972 32693 tablet_bootstrap.cc:492] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:12.041240 31948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet a5634154c6f149d3b2d36b961aee38e0
I20260809 14:30:12.042943 31948 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 288) took 1039 ms. Trace:
I20260809 14:30:12.045363 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 57ee617d858c42f7b03bd287a33eebe4
I20260809 14:30:12.046787 31945 catalog_manager.cc:6565] ReplaceTablet: tablet 57ee617d858c42f7b03bd287a33eebe4 deleted and replaced by tablet 31f9e3b82fa94588a07af146ed78f0e4
I20260809 14:30:12.047197 31945 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49782 (request call id 282) took 1182 ms. Trace:
I20260809 14:30:12.049052 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet a5634154c6f149d3b2d36b961aee38e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:11 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:12.049532 32296 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ee617d858c42f7b03bd287a33eebe4 with delete_type TABLET_DATA_DELETED (replaced by tablet 31f9e3b82fa94588a07af146ed78f0e4) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:12.048010 32141 tablet_service.cc:1467] Processing CreateTablet for tablet 47089009baac430f9bcecec5dbe87687 (DEFAULT_TABLE table=default.Table_7077d276f84a485c840fded026cca74a [id=15a8f74e5d2d458abd4779617f33f2ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.043253 31948 rpcz_store.cc:276] 0809 14:30:11.003854 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:11.003960 (+ 106us) service_pool.cc:227] Handling call
0809 14:30:11.005356 (+ 1396us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:11.006002 (+ 646us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:11.006258 (+ 256us) catalog_manager.cc:2761] Locking tablets
0809 14:30:11.127145 (+120887us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:12.040215 (+913070us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:12.040780 (+ 565us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:12.040989 (+ 209us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:12.042814 (+ 1825us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:11.203968 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:11.204357 (+ 389us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:11.204384 (+ 27us) write_op.cc:435] Acquired schema lock
0809 14:30:11.204398 (+ 14us) tablet.cc:660] Decoding operations
0809 14:30:11.204810 (+ 412us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:11.204850 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:11.204874 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:11.205236 (+ 362us) tablet.cc:699] Row locks acquired
0809 14:30:11.205252 (+ 16us) write_op.cc:260] PREPARE: finished
0809 14:30:11.205361 (+ 109us) write_op.cc:270] Start()
0809 14:30:11.205443 (+ 82us) write_op.cc:276] Timestamp: P: 1786285811205296 usec, L: 0
0809 14:30:11.205461 (+ 18us) op_driver.cc:331] REPLICATION: starting
0809 14:30:11.349326 (+143865us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400012398. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4c2310c4 00007ffa4b4dd4f9 00007ffa4b498f91 00007ffa4b498c6a 00007ffa4b504cd5 00007ffa4b505179 00007ffa4b966ade 00007ffa4b9662b8 00007ffa4b96aba1 00007ffa4b96ab59 00007ffa4b96aae9
0809 14:30:11.349751 (+ 425us) log.cc:842] Serialized 762 byte log entry
0809 14:30:11.733837 (+384086us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b58000704b0. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b483834 00007ffa4b488771 00007ffa4b4886f9 00007ffa4b488689 00007ffa4b488651 00007ffa4b48781d 0000000000580784 00000000005805b9 00007ffa445b46b6 00007ffa445b7b11
0809 14:30:11.999117 (+265280us) op_driver.cc:444] REPLICATION: finished
0809 14:30:12.037078 (+ 37961us) write_op.cc:301] APPLY: starting
0809 14:30:12.037179 (+ 101us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:12.037351 (+ 172us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:12.037383 (+ 32us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:12.037785 (+ 402us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:12.037993 (+ 208us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:12.038058 (+ 65us) write_op.cc:312] APPLY: finished
0809 14:30:12.038439 (+ 381us) log.cc:842] Serialized 29 byte log entry
0809 14:30:12.038719 (+ 280us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.038841 (+ 122us) write_op.cc:454] Released schema lock
0809 14:30:12.038894 (+ 53us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:12.039194 (+ 300us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.039221 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":37670,"apply.run_cpu_time_us":2466,"apply.run_wall_time_us":2466,"num_ops":2,"prepare.queue_time_us":75013,"prepare.run_cpu_time_us":4620,"prepare.run_wall_time_us":146177,"raft.queue_time_us":8499,"raft.run_cpu_time_us":6881,"raft.run_wall_time_us":375320,"replication_time_us":793545,"spinlock_wait_cycles":1167984256}]]}
I20260809 14:30:12.047468 31945 rpcz_store.cc:276] 0809 14:30:10.865196 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:10.865337 (+ 141us) service_pool.cc:227] Handling call
0809 14:30:12.047091 (+1181754us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:11.361270 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:11.361614 (+ 344us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:11.361635 (+ 21us) write_op.cc:435] Acquired schema lock
0809 14:30:11.361647 (+ 12us) tablet.cc:660] Decoding operations
0809 14:30:11.361942 (+ 295us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:11.361983 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:11.362010 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:11.362220 (+ 210us) tablet.cc:699] Row locks acquired
0809 14:30:11.362233 (+ 13us) write_op.cc:260] PREPARE: finished
0809 14:30:11.362374 (+ 141us) write_op.cc:270] Start()
0809 14:30:11.362459 (+ 85us) write_op.cc:276] Timestamp: P: 1786285811362328 usec, L: 0
0809 14:30:11.362485 (+ 26us) op_driver.cc:331] REPLICATION: starting
0809 14:30:11.457658 (+ 95173us) log.cc:842] Serialized 628 byte log entry
0809 14:30:11.458451 (+ 793us) spinlock_profiling.cc:243] Waited 94.5 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48e471 00007ffa4b505198 00007ffa4b966ade 00007ffa4b9662b8 00007ffa4b96aba1 00007ffa4b96ab59 00007ffa4b96aae9 00007ffa4b96aab1 00007ffa4b969dad 0000000000580784 00000000005805b9
0809 14:30:11.734415 (+275964us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b58000704b0. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b483834 00007ffa4b488771 00007ffa4b4886f9 00007ffa4b488689 00007ffa4b488651 00007ffa4b48781d 0000000000580784 00000000005805b9 00007ffa445b46b6 00007ffa445b7b11
0809 14:30:11.999517 (+265102us) op_driver.cc:444] REPLICATION: finished
0809 14:30:12.039696 (+ 40179us) write_op.cc:301] APPLY: starting
0809 14:30:12.039773 (+ 77us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:12.039899 (+ 126us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:12.039920 (+ 21us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:12.040613 (+ 693us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:12.040867 (+ 254us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:12.040912 (+ 45us) write_op.cc:312] APPLY: finished
0809 14:30:12.041114 (+ 202us) log.cc:842] Serialized 29 byte log entry
0809 14:30:12.041268 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.041381 (+ 113us) write_op.cc:454] Released schema lock
0809 14:30:12.041419 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:12.041606 (+ 187us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.041634 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":40032,"apply.run_cpu_time_us":2140,"apply.run_wall_time_us":2139,"num_ops":2,"prepare.queue_time_us":158703,"prepare.run_cpu_time_us":4254,"prepare.run_wall_time_us":150902,"raft.queue_time_us":9212,"raft.run_cpu_time_us":3485,"raft.run_wall_time_us":266705,"replication_time_us":636907,"spinlock_wait_cycles":806974080,"thread_start_us":42392,"threads_started":2}]]}
I20260809 14:30:12.068466 787 tablet_replica.cc:330] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:12.069309 787 raft_consensus.cc:2229] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
W20260809 14:30:12.069604 31939 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 71) took 1317 ms (client timeout 1000 ms). Trace:
I20260809 14:30:12.070000 787 pending_rounds.cc:62] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 1 pending ops.
I20260809 14:30:12.070251 787 pending_rounds.cc:68] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626683082264576 op_type: NO_OP noop_request { }
I20260809 14:30:12.070616 787 raft_consensus.cc:2866] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:12.070935 787 raft_consensus.cc:2258] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:12.039783 32691 raft_consensus.cc:348] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.072136 32691 raft_consensus.cc:374] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.072394 32691 raft_consensus.cc:729] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.073498 31938 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:12.073933 787 ts_tablet_manager.cc:1905] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:12.074259 31938 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:12.074993 31938 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:12.077968 31943 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 2a689385d063469595fcae82aa979bb1 (table default.Table_d841d100269948ca8f7fc467c85a3b0d [id=c19378f8e8994ef28e7b0eba079948c3]). Expected version 2 got 0
I20260809 14:30:12.077059 32691 consensus_queue.cc:260] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.030900 32693 tablet_bootstrap.cc:654] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:12.086395 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet a5634154c6f149d3b2d36b961aee38e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5634154c6f149d3b2d36b961aee38e0 already in progress: creating tablet
W20260809 14:30:12.088136 31943 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 55) took 828 ms (client timeout 1000 ms). Trace:
W20260809 14:30:12.088644 31943 rpcz_store.cc:269] 0809 14:30:11.259896 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:11.357049 (+ 97153us) service_pool.cc:227] Handling call
0809 14:30:11.454311 (+ 97262us) spinlock_profiling.cc:243] Waited 87.4 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:12.088027 (+633716us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":200920448}
W20260809 14:30:12.094295 31938 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 45) took 1158 ms (client timeout 1000 ms). Trace:
W20260809 14:30:12.069912 31939 rpcz_store.cc:269] 0809 14:30:10.752112 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:10.758317 (+ 6205us) service_pool.cc:227] Handling call
0809 14:30:10.765269 (+ 6952us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:12.069431 (+1304162us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:11.852342 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:11.859140 (+ 6798us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:11.859164 (+ 24us) write_op.cc:435] Acquired schema lock
0809 14:30:11.859177 (+ 13us) tablet.cc:660] Decoding operations
0809 14:30:11.859414 (+ 237us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:11.859465 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:11.859493 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:11.859658 (+ 165us) tablet.cc:699] Row locks acquired
0809 14:30:11.859672 (+ 14us) write_op.cc:260] PREPARE: finished
0809 14:30:11.859790 (+ 118us) write_op.cc:270] Start()
0809 14:30:11.859873 (+ 83us) write_op.cc:276] Timestamp: P: 1786285811859717 usec, L: 0
0809 14:30:11.859889 (+ 16us) op_driver.cc:331] REPLICATION: starting
0809 14:30:11.860413 (+ 524us) log.cc:842] Serialized 533 byte log entry
0809 14:30:12.059011 (+198598us) op_driver.cc:444] REPLICATION: finished
0809 14:30:12.064360 (+ 5349us) write_op.cc:301] APPLY: starting
0809 14:30:12.064446 (+ 86us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:12.064593 (+ 147us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:12.064709 (+ 116us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:12.065025 (+ 316us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:12.065192 (+ 167us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:12.065254 (+ 62us) write_op.cc:312] APPLY: finished
0809 14:30:12.065456 (+ 202us) log.cc:842] Serialized 23 byte log entry
0809 14:30:12.066224 (+ 768us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.066299 (+ 75us) write_op.cc:454] Released schema lock
0809 14:30:12.066338 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":11645824,"child_traces":[["op",{"apply.queue_time_us":5182,"num_ops":1,"prepare.queue_time_us":621494,"prepare.run_cpu_time_us":1890,"prepare.run_wall_time_us":8307,"replication_time_us":198957}]]}
I20260809 14:30:12.100809 32691 ts_tablet_manager.cc:1428] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.081s user 0.008s sys 0.000s
W20260809 14:30:12.112509 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e455d9c37f99458a84720427851cd836 consensus state has no leader
W20260809 14:30:12.095093 31938 rpcz_store.cc:269] 0809 14:30:10.936289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:10.936544 (+ 255us) service_pool.cc:227] Handling call
0809 14:30:12.094215 (+1157671us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":5}
I20260809 14:30:12.118415 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47089009baac430f9bcecec5dbe87687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.080183 32705 tablet_bootstrap.cc:492] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:12.121755 32705 ts_tablet_manager.cc:1397] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.288s user 0.008s sys 0.008s
I20260809 14:30:12.122481 825 consensus_queue.cc:1035] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:12.126744 32705 raft_consensus.cc:348] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.127509 32705 raft_consensus.cc:374] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.127736 32705 raft_consensus.cc:729] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.131776 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 47089009baac430f9bcecec5dbe87687 (DEFAULT_TABLE table=default.Table_7077d276f84a485c840fded026cca74a [id=15a8f74e5d2d458abd4779617f33f2ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.136287 32147 tablet_service.cc:1467] Processing CreateTablet for tablet ed328e6f95544c25b08075018cb8caf2 (DEFAULT_TABLE table=default.Table_853333f626c34bc18f5a1245314d7e99 [id=504e8c38c3f1441ebe085ccbcbdab567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.136677 32485 tablet_service.cc:1467] Processing CreateTablet for tablet b12d4849dec841a98e7f1a00c7d5f24e (DEFAULT_TABLE table=default.Table_c2edbb238dd341b49bcd5acde282fb46 [id=d5e8f5c4a3624fc4a4be0b4dcca507e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.137313 32484 tablet_service.cc:1467] Processing CreateTablet for tablet ed328e6f95544c25b08075018cb8caf2 (DEFAULT_TABLE table=default.Table_853333f626c34bc18f5a1245314d7e99 [id=504e8c38c3f1441ebe085ccbcbdab567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.138430 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed328e6f95544c25b08075018cb8caf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.139320 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 24b10fe71bb44ea1ae82d0232c75daf2 (DEFAULT_TABLE table=default.Table_e5e6a1c2957d4ebfaa299a5bf66454e0 [id=98f3952dbc8a4bb0a950019a58fb1d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.139789 32693 tablet_bootstrap.cc:492] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:12.140141 32693 ts_tablet_manager.cc:1397] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.114s user 0.009s sys 0.006s
I20260809 14:30:12.137444 32148 tablet_service.cc:1467] Processing CreateTablet for tablet b12d4849dec841a98e7f1a00c7d5f24e (DEFAULT_TABLE table=default.Table_c2edbb238dd341b49bcd5acde282fb46 [id=d5e8f5c4a3624fc4a4be0b4dcca507e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.140355 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b10fe71bb44ea1ae82d0232c75daf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.140903 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b12d4849dec841a98e7f1a00c7d5f24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.145398 787 ts_tablet_manager.cc:1918] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:12.145809 787 log.cc:1196] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/57ee617d858c42f7b03bd287a33eebe4
I20260809 14:30:12.146652 787 ts_tablet_manager.cc:1939] T 57ee617d858c42f7b03bd287a33eebe4 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:12.051672 32310 tablet_service.cc:1467] Processing CreateTablet for tablet b12d4849dec841a98e7f1a00c7d5f24e (DEFAULT_TABLE table=default.Table_c2edbb238dd341b49bcd5acde282fb46 [id=d5e8f5c4a3624fc4a4be0b4dcca507e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.152971 32693 raft_consensus.cc:348] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.154176 32693 raft_consensus.cc:374] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.156973 32693 raft_consensus.cc:729] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.158008 32693 consensus_queue.cc:260] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.160182 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 24b10fe71bb44ea1ae82d0232c75daf2 (DEFAULT_TABLE table=default.Table_e5e6a1c2957d4ebfaa299a5bf66454e0 [id=98f3952dbc8a4bb0a950019a58fb1d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.162232 32693 ts_tablet_manager.cc:1428] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.022s user 0.005s sys 0.003s
I20260809 14:30:12.164094 32489 tablet_service.cc:1467] Processing CreateTablet for tablet d9deca4f10b548ea9aff5cfdbccc91ab (DEFAULT_TABLE table=default.Table_b342ca5d31c14a89baa7ed0ea2bf8da8 [id=6efa71f406734c998668150f80e73f12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.171285 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b12d4849dec841a98e7f1a00c7d5f24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.174008 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed328e6f95544c25b08075018cb8caf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.174422 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b12d4849dec841a98e7f1a00c7d5f24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.179490 32693 tablet_bootstrap.cc:492] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:12.180367 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b10fe71bb44ea1ae82d0232c75daf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.182691 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 57ee617d858c42f7b03bd287a33eebe4 (table default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]) successfully deleted
I20260809 14:30:12.185473 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47089009baac430f9bcecec5dbe87687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.187628 32691 tablet_bootstrap.cc:492] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:12.188804 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9deca4f10b548ea9aff5cfdbccc91ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.221374 32691 tablet_bootstrap.cc:654] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.208256 32693 tablet_bootstrap.cc:654] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.255254 32691 tablet_bootstrap.cc:492] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:12.255678 32691 ts_tablet_manager.cc:1397] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.068s user 0.012s sys 0.001s
I20260809 14:30:12.279723 32693 tablet_bootstrap.cc:492] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:12.280365 32693 ts_tablet_manager.cc:1397] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.101s user 0.012s sys 0.004s
I20260809 14:30:12.258862 32691 raft_consensus.cc:348] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.281694 32691 raft_consensus.cc:374] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.282034 32691 raft_consensus.cc:729] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.282764 32691 consensus_queue.cc:260] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.286891 32693 raft_consensus.cc:348] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.287796 32693 raft_consensus.cc:374] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.288394 32693 raft_consensus.cc:729] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.300860 32693 consensus_queue.cc:260] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.321508 31946 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:49782 (request call id 294) took 1068 ms. Trace:
I20260809 14:30:12.324631 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 29cd0861b04747699ab2d13cb89b418d
I20260809 14:30:12.332520 32693 ts_tablet_manager.cc:1428] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.052s user 0.007s sys 0.003s
I20260809 14:30:12.128396 32705 consensus_queue.cc:260] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.342253 32693 tablet_bootstrap.cc:492] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:12.348875 31937 catalog_manager.cc:6565] ReplaceTablet: tablet 29cd0861b04747699ab2d13cb89b418d deleted and replaced by tablet 5ad5bf5ed5cc4c66a4c87fc21209c555
I20260809 14:30:12.349257 31937 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49782 (request call id 295) took 1089 ms. Trace:
I20260809 14:30:12.349362 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 29cd0861b04747699ab2d13cb89b418d with delete_type TABLET_DATA_DELETED (replaced by tablet 5ad5bf5ed5cc4c66a4c87fc21209c555) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:12.351415 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 29cd0861b04747699ab2d13cb89b418d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29cd0861b04747699ab2d13cb89b418d already in progress: creating tablet
I20260809 14:30:12.349519 31937 rpcz_store.cc:276] 0809 14:30:11.260095 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:11.547937 (+287842us) service_pool.cc:227] Handling call
0809 14:30:11.858537 (+310600us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4dbb81 00007ffa4b036d37 00007ffa4b036cc2 00007ffa4b036bf1 00007ffa4b036b6c 00007ffa4b035e22 00007ffa492b894c 00007ffa492b7d86
0809 14:30:12.349190 (+490653us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:11.872962 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:11.873276 (+ 314us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:11.873296 (+ 20us) write_op.cc:435] Acquired schema lock
0809 14:30:11.873307 (+ 11us) tablet.cc:660] Decoding operations
0809 14:30:11.873576 (+ 269us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:11.873609 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:11.873629 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:11.873870 (+ 241us) tablet.cc:699] Row locks acquired
0809 14:30:11.873884 (+ 14us) write_op.cc:260] PREPARE: finished
0809 14:30:11.873975 (+ 91us) write_op.cc:270] Start()
0809 14:30:11.874044 (+ 69us) write_op.cc:276] Timestamp: P: 1786285811873905 usec, L: 0
0809 14:30:11.874058 (+ 14us) op_driver.cc:331] REPLICATION: starting
0809 14:30:11.874505 (+ 447us) log.cc:842] Serialized 628 byte log entry
0809 14:30:12.098989 (+224484us) op_driver.cc:444] REPLICATION: finished
0809 14:30:12.129016 (+ 30027us) write_op.cc:301] APPLY: starting
0809 14:30:12.129139 (+ 123us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:12.129365 (+ 226us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:12.129642 (+ 277us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:12.156185 (+ 26543us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:12.156695 (+ 510us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:12.156795 (+ 100us) write_op.cc:312] APPLY: finished
0809 14:30:12.157262 (+ 467us) log.cc:842] Serialized 29 byte log entry
0809 14:30:12.319252 (+161990us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.319400 (+ 148us) write_op.cc:454] Released schema lock
0809 14:30:12.319468 (+ 68us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:12.319976 (+ 508us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.320034 (+ 58us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":711446912,"child_traces":[["op",{"apply.queue_time_us":29756,"apply.run_cpu_time_us":3449,"apply.run_wall_time_us":191332,"num_ops":2,"prepare.queue_time_us":1792,"prepare.run_cpu_time_us":1705,"prepare.run_wall_time_us":3438,"replication_time_us":224742,"threads_started":1}]]}
I20260809 14:30:12.353735 32136 tablet_service.cc:1467] Processing CreateTablet for tablet d9deca4f10b548ea9aff5cfdbccc91ab (DEFAULT_TABLE table=default.Table_b342ca5d31c14a89baa7ed0ea2bf8da8 [id=6efa71f406734c998668150f80e73f12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:12.326061 31946 rpcz_store.cc:276] 0809 14:30:11.251833 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:11.253554 (+ 1721us) service_pool.cc:227] Handling call
0809 14:30:11.466155 (+212601us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9156 00007ffa4b016034 00007ffa4b015fc2 00007ffa4b015ef1 00007ffa4b015e6c 00007ffa4b015122 00007ffa492b894c 00007ffa492b7d86
0809 14:30:11.573078 (+106923us) catalog_manager.cc:2075] Acquired catalog manager lock
0809 14:30:11.576170 (+ 3092us) catalog_manager.cc:2121] Created new table and tablet info
0809 14:30:12.319192 (+743022us) catalog_manager.cc:2181] Wrote table and tablets to system table
0809 14:30:12.319992 (+ 800us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0809 14:30:12.320678 (+ 686us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:11.862330 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:11.862664 (+ 334us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:11.862684 (+ 20us) write_op.cc:435] Acquired schema lock
0809 14:30:11.862695 (+ 11us) tablet.cc:660] Decoding operations
0809 14:30:11.862921 (+ 226us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:11.862951 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:11.862968 (+ 17us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:11.863140 (+ 172us) tablet.cc:699] Row locks acquired
0809 14:30:11.863158 (+ 18us) write_op.cc:260] PREPARE: finished
0809 14:30:11.863249 (+ 91us) write_op.cc:270] Start()
0809 14:30:11.863317 (+ 68us) write_op.cc:276] Timestamp: P: 1786285811863181 usec, L: 0
0809 14:30:11.863332 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 14:30:11.863863 (+ 531us) log.cc:842] Serialized 433 byte log entry
0809 14:30:12.098585 (+234722us) op_driver.cc:444] REPLICATION: finished
0809 14:30:12.111092 (+ 12507us) write_op.cc:301] APPLY: starting
0809 14:30:12.111172 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:12.111551 (+ 379us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:12.111713 (+ 162us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:12.112500 (+ 787us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:12.112663 (+ 163us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 14:30:12.112725 (+ 62us) write_op.cc:312] APPLY: finished
0809 14:30:12.112928 (+ 203us) log.cc:842] Serialized 29 byte log entry
0809 14:30:12.317934 (+205006us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.318173 (+ 239us) write_op.cc:454] Released schema lock
0809 14:30:12.318237 (+ 64us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:12.318709 (+ 472us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.318755 (+ 46us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":484994432,"child_traces":[["op",{"apply.queue_time_us":12233,"apply.run_cpu_time_us":2920,"apply.run_wall_time_us":207970,"num_ops":2,"prepare.queue_time_us":203502,"prepare.run_cpu_time_us":1684,"prepare.run_wall_time_us":1682,"replication_time_us":235100}]]}
I20260809 14:30:12.355581 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9deca4f10b548ea9aff5cfdbccc91ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:12.373378 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet a5634154c6f149d3b2d36b961aee38e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:11 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:12.375357 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ee617d858c42f7b03bd287a33eebe4 with delete_type TABLET_DATA_DELETED (replaced by tablet 31f9e3b82fa94588a07af146ed78f0e4) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:12.378103 32693 tablet_bootstrap.cc:654] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.379912 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 29cd0861b04747699ab2d13cb89b418d with delete_type TABLET_DATA_DELETED (replaced by tablet 5ad5bf5ed5cc4c66a4c87fc21209c555) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:12.337163 735 consensus_queue.cc:1035] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.059s
I20260809 14:30:12.406704 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet a5634154c6f149d3b2d36b961aee38e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:11 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:12.407964 32479 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ee617d858c42f7b03bd287a33eebe4 with delete_type TABLET_DATA_DELETED (replaced by tablet 31f9e3b82fa94588a07af146ed78f0e4) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:12.409137 32481 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 1eb9d7c27381473aa87b718cca57b678) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:12.410378 32478 tablet_service.cc:1514] Processing DeleteTablet for tablet 29cd0861b04747699ab2d13cb89b418d with delete_type TABLET_DATA_DELETED (replaced by tablet 5ad5bf5ed5cc4c66a4c87fc21209c555) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:12.414559 32693 tablet_bootstrap.cc:492] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:12.414966 32693 ts_tablet_manager.cc:1397] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.073s user 0.012s sys 0.000s
I20260809 14:30:12.418772 32693 raft_consensus.cc:348] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.425333 32693 raft_consensus.cc:374] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.425582 32693 raft_consensus.cc:729] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.426721 32326 raft_consensus.cc:3037] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.432753 32693 consensus_queue.cc:260] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.440861 32693 ts_tablet_manager.cc:1428] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.026s user 0.004s sys 0.005s
I20260809 14:30:12.441828 32693 tablet_bootstrap.cc:492] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:12.457335 32693 tablet_bootstrap.cc:654] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.464604 589 raft_consensus.cc:482] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:12.465303 589 raft_consensus.cc:504] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.469048 589 leader_election.cc:288] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:12.471207 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:12.472934 32512 raft_consensus.cc:2454] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:12.474716 32260 leader_election.cc:302] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:12.476142 589 raft_consensus.cc:2781] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:12.476707 589 raft_consensus.cc:482] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:12.477257 589 raft_consensus.cc:3037] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.488184 753 raft_consensus.cc:482] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:12.489647 32691 ts_tablet_manager.cc:1428] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.234s user 0.003s sys 0.006s
I20260809 14:30:12.489051 753 raft_consensus.cc:504] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.491437 32691 tablet_bootstrap.cc:492] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:12.493528 753 leader_election.cc:288] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:12.494809 673 raft_consensus.cc:482] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:12.495548 673 raft_consensus.cc:504] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.501226 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9f01d25eac34d4d998dc457be6a5d9b" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:12.502336 32330 raft_consensus.cc:2454] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:30:12.502473 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:12.505079 32512 raft_consensus.cc:2454] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:12.507740 32260 leader_election.cc:302] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:12.508575 753 raft_consensus.cc:2781] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:12.509119 753 raft_consensus.cc:482] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:12.509721 753 raft_consensus.cc:3037] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.506273 32443 leader_election.cc:302] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:12.512977 818 raft_consensus.cc:2781] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:12.513622 818 raft_consensus.cc:482] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:12.514315 818 raft_consensus.cc:3037] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.518923 32705 ts_tablet_manager.cc:1428] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.397s user 0.006s sys 0.003s
I20260809 14:30:12.519173 754 consensus_queue.cc:1035] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:12.500617 673 leader_election.cc:288] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:12.556864 843 tablet_replica.cc:330] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:12.557984 843 raft_consensus.cc:2229] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:12.558884 32693 tablet_bootstrap.cc:492] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:12.558835 843 raft_consensus.cc:2258] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:12.559458 32693 ts_tablet_manager.cc:1397] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.118s user 0.007s sys 0.016s
I20260809 14:30:12.560372 32330 raft_consensus.cc:3037] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.578037 32693 raft_consensus.cc:348] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.578768 32693 raft_consensus.cc:374] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.579006 32693 raft_consensus.cc:729] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.584832 853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:30:12.586534 853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:12.593340 32705 tablet_bootstrap.cc:492] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:12.594683 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet a5634154c6f149d3b2d36b961aee38e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5634154c6f149d3b2d36b961aee38e0 already in progress: creating tablet
I20260809 14:30:12.594550 32693 consensus_queue.cc:260] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.582756 843 ts_tablet_manager.cc:1905] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:12.606950 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet a5634154c6f149d3b2d36b961aee38e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5634154c6f149d3b2d36b961aee38e0 already in progress: creating tablet
I20260809 14:30:12.630308 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 7 candidate_status { last_received { term: 6 index: 320 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:30:12.628466 853 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:30:12.637115 854 raft_consensus.cc:482] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:12.637931 854 raft_consensus.cc:504] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.642936 589 raft_consensus.cc:504] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.646529 840 consensus_queue.cc:1035] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 14:30:12.653093 32135 consensus_meta.cc:287] T 24b10fe71bb44ea1ae82d0232c75daf2 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.504s user 0.006s sys 0.003s
I20260809 14:30:12.659898 32693 ts_tablet_manager.cc:1428] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.100s user 0.011s sys 0.004s
W20260809 14:30:12.662016 31933 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a0b196e429514cbc803e9a99a986d4d2 (table default.Table_42a687657ecc485a9c3d956e245784c5 [id=58e17b048a594020a069730291d81345], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a0b196e429514cbc803e9a99a986d4d2 consensus state has no leader
I20260809 14:30:12.661077 854 leader_election.cc:288] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:12.665828 32693 tablet_bootstrap.cc:492] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
W20260809 14:30:12.675930 31840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
I20260809 14:30:12.686205 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9f01d25eac34d4d998dc457be6a5d9b" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:12.649981 753 raft_consensus.cc:504] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.692021 753 leader_election.cc:288] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:12.693526 587 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.593s user 0.001s sys 0.013s
W20260809 14:30:12.694231 587 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.595s user 0.002s sys 0.013s
I20260809 14:30:12.702533 31944 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:49782 (request call id 296) took 1427 ms. Trace:
I20260809 14:30:12.675135 589 leader_election.cc:288] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:12.702847 31944 rpcz_store.cc:276] 0809 14:30:11.274012 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:11.550164 (+276152us) service_pool.cc:227] Handling call
0809 14:30:11.854312 (+304148us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9156 00007ffa4b016034 00007ffa4b015fc2 00007ffa4b015ef1 00007ffa4b015e6c 00007ffa4b015122 00007ffa492b894c 00007ffa492b7d86
0809 14:30:11.856407 (+ 2095us) catalog_manager.cc:2075] Acquired catalog manager lock
0809 14:30:11.871413 (+ 15006us) catalog_manager.cc:2121] Created new table and tablet info
0809 14:30:12.700559 (+829146us) catalog_manager.cc:2181] Wrote table and tablets to system table
0809 14:30:12.701207 (+ 648us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0809 14:30:12.701830 (+ 623us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:11.876789 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:11.877217 (+ 428us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:11.877240 (+ 23us) write_op.cc:435] Acquired schema lock
0809 14:30:11.877252 (+ 12us) tablet.cc:660] Decoding operations
0809 14:30:11.877509 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:11.877540 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:11.877573 (+ 33us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:11.877770 (+ 197us) tablet.cc:699] Row locks acquired
0809 14:30:11.877782 (+ 12us) write_op.cc:260] PREPARE: finished
0809 14:30:11.877883 (+ 101us) write_op.cc:270] Start()
0809 14:30:11.877953 (+ 70us) write_op.cc:276] Timestamp: P: 1786285811877812 usec, L: 0
0809 14:30:11.877983 (+ 30us) op_driver.cc:331] REPLICATION: starting
0809 14:30:11.878685 (+ 702us) log.cc:842] Serialized 433 byte log entry
0809 14:30:12.695281 (+816596us) op_driver.cc:444] REPLICATION: finished
0809 14:30:12.695860 (+ 579us) write_op.cc:301] APPLY: starting
0809 14:30:12.695972 (+ 112us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:12.696299 (+ 327us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:12.696651 (+ 352us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:12.697562 (+ 911us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:12.697920 (+ 358us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 14:30:12.698009 (+ 89us) write_op.cc:312] APPLY: finished
0809 14:30:12.698510 (+ 501us) log.cc:842] Serialized 29 byte log entry
0809 14:30:12.699838 (+ 1328us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:12.700010 (+ 172us) write_op.cc:454] Released schema lock
0809 14:30:12.700079 (+ 69us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":697215232,"child_traces":[["op",{"apply.queue_time_us":188,"apply.run_cpu_time_us":3942,"apply.run_wall_time_us":9332,"num_ops":2,"prepare.queue_time_us":3889,"prepare.run_cpu_time_us":2164,"prepare.run_wall_time_us":120617,"replication_time_us":817062}]]}
W20260809 14:30:12.732168 31840 leader_election.cc:334] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223): Network error: Client connection negotiation failed: client connection to 127.30.252.126:43223: connect: Connection refused (error 111)
I20260809 14:30:12.734273 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852dbd0962ae4211923851c7a632768e; no voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:30:12.735913 853 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 14:30:12.740152 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:12.744568 32693 tablet_bootstrap.cc:654] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.754895 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:12.755112 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:12.755540 32168 raft_consensus.cc:2454] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:12.754895 32166 raft_consensus.cc:3037] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.755059 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:12.762794 32705 tablet_bootstrap.cc:654] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:12.765437 32260 leader_election.cc:341] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:12.766443 32260 leader_election.cc:341] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:12.767314 32260 leader_election.cc:302] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
W20260809 14:30:12.768186 32260 leader_election.cc:341] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:12.769080 589 raft_consensus.cc:2781] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:30:12.769477 589 raft_consensus.cc:482] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:12.769851 589 raft_consensus.cc:3037] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:12.779613 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:12.787824 32260 leader_election.cc:341] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:12.797891 589 raft_consensus.cc:504] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.785604 847 tablet_replica.cc:330] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:12.805040 32693 tablet_bootstrap.cc:492] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:12.805472 32693 ts_tablet_manager.cc:1397] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.140s user 0.014s sys 0.003s
I20260809 14:30:12.807487 589 leader_election.cc:288] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:12.808147 32693 raft_consensus.cc:348] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.808758 32693 raft_consensus.cc:374] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.809130 32693 raft_consensus.cc:729] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.809793 32693 consensus_queue.cc:260] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.812184 32691 tablet_bootstrap.cc:654] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.813311 32693 ts_tablet_manager.cc:1428] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260809 14:30:12.815017 32693 tablet_bootstrap.cc:492] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:12.816867 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:12.818332 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:12.818990 32513 raft_consensus.cc:3037] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.821673 32693 tablet_bootstrap.cc:654] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.821959 32509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:12.822559 32509 raft_consensus.cc:3037] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.824944 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:12.747932 673 consensus_queue.cc:1035] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260809 14:30:12.835417 847 raft_consensus.cc:2229] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:12.836030 847 raft_consensus.cc:2258] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:12.838215 847 ts_tablet_manager.cc:1905] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:12.843475 818 raft_consensus.cc:504] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.856815 32693 tablet_bootstrap.cc:492] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:12.857251 32693 ts_tablet_manager.cc:1397] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.043s user 0.022s sys 0.014s
I20260809 14:30:12.860041 32693 raft_consensus.cc:348] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.860773 32693 raft_consensus.cc:374] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.861018 32693 raft_consensus.cc:729] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.861603 32693 consensus_queue.cc:260] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.864660 589 raft_consensus.cc:482] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:12.865087 589 raft_consensus.cc:504] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.867018 589 leader_election.cc:288] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:12.864351 32693 ts_tablet_manager.cc:1428] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 14:30:12.869694 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c4cbdda4ed48a8b34728d231497d02" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:12.870328 32508 raft_consensus.cc:2454] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:12.870465 32693 tablet_bootstrap.cc:492] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:12.871626 32260 leader_election.cc:302] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:12.872565 753 raft_consensus.cc:2781] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:12.872954 753 raft_consensus.cc:482] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:12.873277 753 raft_consensus.cc:3037] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:12.878439 818 leader_election.cc:288] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:12.879879 32693 tablet_bootstrap.cc:654] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.883170 589 consensus_queue.cc:1035] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:12.892323 32691 tablet_bootstrap.cc:492] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:12.924170 32691 ts_tablet_manager.cc:1397] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.433s user 0.012s sys 0.001s
I20260809 14:30:12.926550 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c4cbdda4ed48a8b34728d231497d02" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:12.926649 32691 raft_consensus.cc:348] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.927315 32691 raft_consensus.cc:374] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:12.927533 32691 raft_consensus.cc:729] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.928046 32691 consensus_queue.cc:260] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.930083 32691 ts_tablet_manager.cc:1428] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 14:30:12.930882 32691 tablet_bootstrap.cc:492] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:30:12.932503 32260 leader_election.cc:341] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:12.935130 32509 raft_consensus.cc:2454] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:12.935770 32691 tablet_bootstrap.cc:654] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:12.939260 32513 raft_consensus.cc:2454] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:12.928490 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:12.944659 32170 raft_consensus.cc:3037] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:12.975373 753 raft_consensus.cc:504] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.976876 589 raft_consensus.cc:482] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:12.977363 589 raft_consensus.cc:504] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.979238 589 leader_election.cc:288] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:12.972038 32260 leader_election.cc:302] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:12.994197 32260 leader_election.cc:302] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:12.995122 852 raft_consensus.cc:2781] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:12.995596 852 raft_consensus.cc:686] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:12.996446 852 consensus_queue.cc:237] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:12.999831 860 raft_consensus.cc:2781] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:13.000298 860 raft_consensus.cc:686] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:13.000939 793 raft_consensus.cc:482] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:13.001137 860 consensus_queue.cc:237] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.001482 793 raft_consensus.cc:504] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.005318 793 leader_election.cc:288] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:13.006990 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:13.012542 843 ts_tablet_manager.cc:1918] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:13.013116 843 log.cc:1196] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/57ee617d858c42f7b03bd287a33eebe4
W20260809 14:30:13.023909 31735 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 69904.8us
I20260809 14:30:13.033708 843 ts_tablet_manager.cc:1939] T 57ee617d858c42f7b03bd287a33eebe4 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:13.035629 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 57ee617d858c42f7b03bd287a33eebe4 (table default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]) successfully deleted
W20260809 14:30:13.037978 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 349749us
W20260809 14:30:13.045361 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 115245us
I20260809 14:30:13.018707 32170 raft_consensus.cc:2454] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 2.
I20260809 14:30:13.051858 822 raft_consensus.cc:482] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:13.052486 822 raft_consensus.cc:504] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.061597 822 leader_election.cc:288] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:13.069088 32329 raft_consensus.cc:1226] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Rejecting Update request from peer 6bed14aff53a41b2bbf5a13ce55dec5a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260809 14:30:13.075101 32705 tablet_bootstrap.cc:492] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:13.075501 32705 ts_tablet_manager.cc:1397] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.482s user 0.007s sys 0.017s
W20260809 14:30:13.079774 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 29cd0861b04747699ab2d13cb89b418d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29cd0861b04747699ab2d13cb89b418d already in progress: creating tablet
I20260809 14:30:13.081328 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
W20260809 14:30:13.081869 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 87780.4us
I20260809 14:30:13.093041 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9f01d25eac34d4d998dc457be6a5d9b" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:13.104635 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9f01d25eac34d4d998dc457be6a5d9b" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:13.107730 32095 leader_election.cc:302] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:13.105451 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:13.105496 32329 raft_consensus.cc:3037] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.113159 32705 raft_consensus.cc:348] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.113854 32705 raft_consensus.cc:374] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.114131 32705 raft_consensus.cc:729] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:12.978124 753 leader_election.cc:288] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:13.118008 822 consensus_queue.cc:1046] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:13.119971 818 raft_consensus.cc:3032] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 14:30:13.120296 818 raft_consensus.cc:729] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:13.121227 818 consensus_queue.cc:260] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.122443 818 raft_consensus.cc:3037] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:13.123606 888 raft_consensus.cc:2726] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:13.135047 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c4cbdda4ed48a8b34728d231497d02" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:13.136544 32439 leader_election.cc:341] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:13.137573 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_373053e6ca574ffa8cee29e912be22da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:13.140075 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_373053e6ca574ffa8cee29e912be22da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:13.141777 31939 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ed937fcc529a4bad9edc6085fdf5b47d" } new_table_name: "default.Table_eae0b4d8859f4817bbf3aa3f21d3a9ce" modify_external_catalogs: true
I20260809 14:30:13.144115 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3f84f78f94f744ae9a10e20f3b6248ef" } modify_external_catalogs: true
I20260809 14:30:13.145614 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e6905fa41eb64a4c9eeabed102a72d23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:13.147043 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6905fa41eb64a4c9eeabed102a72d23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:13.154284 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_250ec22f736f4ef885234ffff62f2054" } modify_external_catalogs: true
I20260809 14:30:13.159729 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_423b0a16be834083833ae85a541ff1c4" } new_table_name: "default.Table_02301a82edfd45bd9511edf923e1028d" modify_external_catalogs: true
I20260809 14:30:13.161202 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f13fa07f50084edc949a28f2c35bd624"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:13.162891 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f13fa07f50084edc949a28f2c35bd624 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:13.165719 32439 leader_election.cc:341] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:13.164839 754 raft_consensus.cc:482] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:13.141515 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c4cbdda4ed48a8b34728d231497d02" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:13.167821 32515 raft_consensus.cc:3037] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.168156 754 raft_consensus.cc:504] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.170213 754 leader_election.cc:288] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:13.171494 32329 raft_consensus.cc:2454] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:13.150200 32691 tablet_bootstrap.cc:492] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:13.172778 32691 ts_tablet_manager.cc:1397] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.242s user 0.021s sys 0.020s
I20260809 14:30:13.175364 32691 raft_consensus.cc:348] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.176056 32691 raft_consensus.cc:374] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.176324 32691 raft_consensus.cc:729] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.177078 32691 consensus_queue.cc:260] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:13.116259 32443 leader_election.cc:341] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:13.178335 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 62112.9us
I20260809 14:30:13.186160 32443 leader_election.cc:302] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:13.188657 32260 consensus_peers.cc:487] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:13.188843 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3f84f78f94f744ae9a10e20f3b6248ef" } modify_external_catalogs: true
W20260809 14:30:13.189028 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 45849.8us
I20260809 14:30:13.189743 754 consensus_queue.cc:1035] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:13.190747 32671 rpc_server.cc:307] RPC server started. Bound to: 127.30.252.126:43223
I20260809 14:30:13.192638 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_250ec22f736f4ef885234ffff62f2054" } modify_external_catalogs: true
I20260809 14:30:13.194917 32671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/master-0/data/info.pb
I20260809 14:30:13.196750 32260 leader_election.cc:302] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:13.196856 818 raft_consensus.cc:2781] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:13.197418 754 raft_consensus.cc:2781] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Leader election won for term 2
I20260809 14:30:13.197419 818 raft_consensus.cc:686] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:13.198215 754 raft_consensus.cc:686] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:13.198345 818 consensus_queue.cc:237] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.199142 754 consensus_queue.cc:237] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.199905 31729 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyetPbO/build/tsan/bin/kudu as pid 32671
I20260809 14:30:13.207852 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:13.208161 32691 ts_tablet_manager.cc:1428] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260809 14:30:13.208637 32508 raft_consensus.cc:2454] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:13.219873 32691 tablet_bootstrap.cc:492] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:13.239547 32515 raft_consensus.cc:2454] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:13.241885 847 ts_tablet_manager.cc:1918] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:13.242132 32260 leader_election.cc:302] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:13.244426 852 raft_consensus.cc:2781] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:13.245025 852 raft_consensus.cc:482] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:13.245479 852 raft_consensus.cc:3037] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.203378 31945 catalog_manager.cc:5495] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65). New cstate: current_term: 1 leader_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:13.245244 847 log.cc:1196] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/57ee617d858c42f7b03bd287a33eebe4
I20260809 14:30:13.249125 31945 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:13.250047 847 ts_tablet_manager.cc:1939] T 57ee617d858c42f7b03bd287a33eebe4 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:13.249948 31945 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:13.250366 32693 tablet_bootstrap.cc:492] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:13.250756 32693 ts_tablet_manager.cc:1397] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.381s user 0.015s sys 0.014s
I20260809 14:30:13.250998 31945 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:13.255108 32693 raft_consensus.cc:348] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.257294 898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.252.126:43223 every 8 connection(s)
I20260809 14:30:13.257656 32693 raft_consensus.cc:374] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.257905 32693 raft_consensus.cc:729] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.261680 32693 consensus_queue.cc:260] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.265798 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc8d0cad33b4778af9fe9247743c38f" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:13.266494 32510 raft_consensus.cc:2454] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:13.267767 32693 ts_tablet_manager.cc:1428] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260809 14:30:13.271024 32693 tablet_bootstrap.cc:492] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:13.279381 32693 tablet_bootstrap.cc:654] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.263043 32260 leader_election.cc:302] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:13.114900 32705 consensus_queue.cc:260] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.282294 32705 ts_tablet_manager.cc:1428] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.207s user 0.002s sys 0.004s
I20260809 14:30:13.283505 754 raft_consensus.cc:482] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:13.283987 754 raft_consensus.cc:504] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.285910 754 leader_election.cc:288] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:13.289554 32705 tablet_bootstrap.cc:492] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:13.293413 32260 leader_election.cc:302] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:13.295275 753 raft_consensus.cc:2781] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:13.295480 754 raft_consensus.cc:2781] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:13.295843 753 raft_consensus.cc:686] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:13.296801 753 consensus_queue.cc:237] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.295852 754 raft_consensus.cc:482] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:13.304312 754 raft_consensus.cc:3037] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.310521 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 57ee617d858c42f7b03bd287a33eebe4 (table default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]) successfully deleted
I20260809 14:30:13.311901 847 tablet_replica.cc:330] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:13.312490 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:13.313887 847 raft_consensus.cc:2229] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:13.317523 32691 tablet_bootstrap.cc:654] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:13.311369 32260 leader_election.cc:341] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:13.334311 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc8d0cad33b4778af9fe9247743c38f" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:13.335577 32166 raft_consensus.cc:2454] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:13.340647 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:13.341364 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:13.341854 32510 raft_consensus.cc:2454] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:13.345561 32260 leader_election.cc:302] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:13.350133 589 raft_consensus.cc:2781] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:13.350791 589 raft_consensus.cc:482] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:13.351418 589 raft_consensus.cc:3037] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.349196 852 raft_consensus.cc:504] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.364957 852 leader_election.cc:288] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:13.366139 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:13.367115 754 raft_consensus.cc:504] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.372524 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:13.373243 847 raft_consensus.cc:2258] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:13.373960 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:13.377584 32705 tablet_bootstrap.cc:654] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:13.379917 32439 leader_election.cc:341] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:13.380430 32439 leader_election.cc:302] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:13.382411 818 raft_consensus.cc:2726] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:13.368251 32260 leader_election.cc:341] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:13.396623 32260 leader_election.cc:341] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:13.397440 32260 leader_election.cc:341] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:13.368358 589 raft_consensus.cc:504] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:13.398319 32260 leader_election.cc:341] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:13.403662 32260 leader_election.cc:302] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:13.409780 852 raft_consensus.cc:2726] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 14:30:13.414530 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:13.415612 32510 raft_consensus.cc:3037] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.423206 32693 tablet_bootstrap.cc:492] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:13.423787 32693 ts_tablet_manager.cc:1397] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.153s user 0.011s sys 0.006s
I20260809 14:30:13.423619 754 leader_election.cc:288] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:13.424763 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc8d0cad33b4778af9fe9247743c38f" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:13.425591 32515 raft_consensus.cc:3037] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.427696 32691 tablet_bootstrap.cc:492] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:13.428162 32691 ts_tablet_manager.cc:1397] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.209s user 0.016s sys 0.027s
I20260809 14:30:13.429284 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:13.429612 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc8d0cad33b4778af9fe9247743c38f" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:13.430555 32166 raft_consensus.cc:3037] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.434880 32693 raft_consensus.cc:348] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.435665 32693 raft_consensus.cc:374] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.435969 32693 raft_consensus.cc:729] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:30:13.442057 32260 leader_election.cc:341] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:13.442382 32691 raft_consensus.cc:348] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.443302 32691 raft_consensus.cc:374] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.443643 32691 raft_consensus.cc:729] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.440109 32693 consensus_queue.cc:260] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.401353 589 leader_election.cc:288] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:13.452116 32693 ts_tablet_manager.cc:1428] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20260809 14:30:13.453266 32693 tablet_bootstrap.cc:492] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:13.456547 32705 tablet_bootstrap.cc:492] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:13.456955 32705 ts_tablet_manager.cc:1397] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.168s user 0.015s sys 0.007s
I20260809 14:30:13.460827 32691 consensus_queue.cc:260] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.463073 32691 ts_tablet_manager.cc:1428] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260809 14:30:13.463582 31938 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:49782 (request call id 301) took 1196 ms. Trace:
I20260809 14:30:13.463855 32691 tablet_bootstrap.cc:492] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:13.464687 32515 raft_consensus.cc:2454] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:13.463994 31938 rpcz_store.cc:276] 0809 14:30:12.267041 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.267252 (+ 211us) service_pool.cc:227] Handling call
0809 14:30:13.137356 (+870104us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9156 00007ffa4b016034 00007ffa4b015fc2 00007ffa4b015ef1 00007ffa4b015e6c 00007ffa4b015122 00007ffa492b894c 00007ffa492b7d86
0809 14:30:13.176583 (+ 39227us) catalog_manager.cc:2075] Acquired catalog manager lock
0809 14:30:13.178850 (+ 2267us) catalog_manager.cc:2121] Created new table and tablet info
0809 14:30:13.462740 (+283890us) catalog_manager.cc:2181] Wrote table and tablets to system table
0809 14:30:13.463149 (+ 409us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0809 14:30:13.463504 (+ 355us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.270106 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.270571 (+ 465us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.270606 (+ 35us) write_op.cc:435] Acquired schema lock
0809 14:30:13.270640 (+ 34us) tablet.cc:660] Decoding operations
0809 14:30:13.270961 (+ 321us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.271032 (+ 71us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.271064 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:13.271273 (+ 209us) tablet.cc:699] Row locks acquired
0809 14:30:13.271299 (+ 26us) write_op.cc:260] PREPARE: finished
0809 14:30:13.271444 (+ 145us) write_op.cc:270] Start()
0809 14:30:13.271556 (+ 112us) write_op.cc:276] Timestamp: P: 1786285813271381 usec, L: 0
0809 14:30:13.271575 (+ 19us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.272213 (+ 638us) log.cc:842] Serialized 433 byte log entry
0809 14:30:13.406615 (+134402us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.445312 (+ 38697us) write_op.cc:301] APPLY: starting
0809 14:30:13.445386 (+ 74us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.445511 (+ 125us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.445646 (+ 135us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.446151 (+ 505us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.446351 (+ 200us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 14:30:13.446414 (+ 63us) write_op.cc:312] APPLY: finished
0809 14:30:13.446668 (+ 254us) log.cc:842] Serialized 29 byte log entry
0809 14:30:13.461786 (+ 15118us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.461885 (+ 99us) write_op.cc:454] Released schema lock
0809 14:30:13.461932 (+ 47us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":1998504320,"child_traces":[["op",{"apply.queue_time_us":38473,"num_ops":2,"prepare.queue_time_us":89905,"prepare.run_cpu_time_us":2469,"prepare.run_wall_time_us":50273,"raft.queue_time_us":165,"raft.run_cpu_time_us":1268,"raft.run_wall_time_us":1268,"replication_time_us":134902,"thread_start_us":139067,"threads_started":2}]]}
I20260809 14:30:13.470054 32705 raft_consensus.cc:348] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.474833 32705 raft_consensus.cc:374] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.475170 32705 raft_consensus.cc:729] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.475832 32705 consensus_queue.cc:260] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.477952 32705 ts_tablet_manager.cc:1428] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260809 14:30:13.478902 32705 tablet_bootstrap.cc:492] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:13.473901 847 ts_tablet_manager.cc:1905] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:13.489686 32260 leader_election.cc:302] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:13.491040 589 raft_consensus.cc:2781] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:13.493925 32510 raft_consensus.cc:2454] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:13.493934 589 raft_consensus.cc:686] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:13.494925 32260 leader_election.cc:302] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:13.496356 754 raft_consensus.cc:2781] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:13.497110 754 raft_consensus.cc:686] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:13.497934 754 consensus_queue.cc:237] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.498476 32166 raft_consensus.cc:2454] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:13.504321 589 consensus_queue.cc:237] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.487222 32691 tablet_bootstrap.cc:654] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.491568 32693 tablet_bootstrap.cc:654] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.513908 32705 tablet_bootstrap.cc:654] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.553316 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 1d52f4b7d49544109df36c085e4aaec3
I20260809 14:30:13.555284 31943 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 299) took 1799 ms. Trace:
I20260809 14:30:13.558378 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d52f4b7d49544109df36c085e4aaec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:13 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:13.560431 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d52f4b7d49544109df36c085e4aaec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:13 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:13.562727 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d52f4b7d49544109df36c085e4aaec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:13 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:13.563149 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 1d52f4b7d49544109df36c085e4aaec3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d52f4b7d49544109df36c085e4aaec3 already in progress: creating tablet
I20260809 14:30:13.555639 31943 rpcz_store.cc:276] 0809 14:30:11.756228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.090265 (+334037us) service_pool.cc:227] Handling call
0809 14:30:13.154167 (+1063902us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9466 00007ffa4b01ab34 00007ffa4b01aac2 00007ffa4b01a9f1 00007ffa4b01a96c 00007ffa4b019c22 00007ffa492b894c 00007ffa492b7d86
0809 14:30:13.193151 (+ 38984us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:13.193774 (+ 623us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:13.193888 (+ 114us) catalog_manager.cc:2761] Locking tablets
0809 14:30:13.194408 (+ 520us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:13.552335 (+357927us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:13.552738 (+ 403us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:13.552887 (+ 149us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:13.555164 (+ 2277us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.320732 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.321764 (+ 1032us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.321812 (+ 48us) write_op.cc:435] Acquired schema lock
0809 14:30:13.321824 (+ 12us) tablet.cc:660] Decoding operations
0809 14:30:13.322417 (+ 593us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.322457 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.322496 (+ 39us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:13.322814 (+ 318us) tablet.cc:699] Row locks acquired
0809 14:30:13.322825 (+ 11us) write_op.cc:260] PREPARE: finished
0809 14:30:13.322944 (+ 119us) write_op.cc:270] Start()
0809 14:30:13.323029 (+ 85us) write_op.cc:276] Timestamp: P: 1786285813322887 usec, L: 0
0809 14:30:13.323046 (+ 17us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.323847 (+ 801us) log.cc:842] Serialized 731 byte log entry
0809 14:30:13.548093 (+224246us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.548546 (+ 453us) write_op.cc:301] APPLY: starting
0809 14:30:13.548630 (+ 84us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.548840 (+ 210us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.549107 (+ 267us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.549756 (+ 649us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.550069 (+ 313us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:13.550158 (+ 89us) write_op.cc:312] APPLY: finished
0809 14:30:13.550453 (+ 295us) log.cc:842] Serialized 29 byte log entry
0809 14:30:13.550678 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.550787 (+ 109us) write_op.cc:454] Released schema lock
0809 14:30:13.550840 (+ 53us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:13.551283 (+ 443us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.551339 (+ 56us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":250,"spinlock_wait_cycles":2443174656,"child_traces":[["op",{"apply.queue_time_us":191,"apply.run_cpu_time_us":3012,"apply.run_wall_time_us":3011,"num_ops":2,"prepare.queue_time_us":124260,"prepare.run_cpu_time_us":3583,"prepare.run_wall_time_us":63607,"replication_time_us":224895}]]}
I20260809 14:30:13.575863 32691 tablet_bootstrap.cc:492] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:13.576365 32691 ts_tablet_manager.cc:1397] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.113s user 0.007s sys 0.003s
I20260809 14:30:13.580904 32693 tablet_bootstrap.cc:492] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:13.581465 32693 ts_tablet_manager.cc:1397] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.128s user 0.015s sys 0.009s
I20260809 14:30:13.584779 32691 raft_consensus.cc:348] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.586220 32691 raft_consensus.cc:374] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.586609 32691 raft_consensus.cc:729] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.589531 32705 tablet_bootstrap.cc:492] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:13.590050 32705 ts_tablet_manager.cc:1397] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.111s user 0.009s sys 0.000s
I20260809 14:30:13.597600 32691 consensus_queue.cc:260] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.606351 32693 raft_consensus.cc:348] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.605221 32705 raft_consensus.cc:348] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.608335 32705 raft_consensus.cc:374] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.608681 32705 raft_consensus.cc:729] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.609323 32705 consensus_queue.cc:260] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.606199 32691 ts_tablet_manager.cc:1428] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.029s user 0.008s sys 0.001s
I20260809 14:30:13.611472 32224 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:30:13.612146 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:13.615471 847 ts_tablet_manager.cc:1918] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:13.616078 847 log.cc:1196] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/29fdaef8a95e4aa0b23018d384442d16
I20260809 14:30:13.617056 847 ts_tablet_manager.cc:1939] T 29fdaef8a95e4aa0b23018d384442d16 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:13.627552 32569 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:30:13.628551 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:13.607157 32693 raft_consensus.cc:374] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.634603 32693 raft_consensus.cc:729] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.637995 32705 ts_tablet_manager.cc:1428] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.048s user 0.009s sys 0.000s
I20260809 14:30:13.636760 32693 consensus_queue.cc:260] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.645859 905 sys_catalog.cc:263] Verifying existing consensus state
W20260809 14:30:13.651139 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 1d52f4b7d49544109df36c085e4aaec3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d52f4b7d49544109df36c085e4aaec3 already in progress: creating tablet
I20260809 14:30:13.653702 32693 ts_tablet_manager.cc:1428] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.072s user 0.005s sys 0.004s
I20260809 14:30:13.654745 32693 tablet_bootstrap.cc:492] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:13.659035 32705 tablet_bootstrap.cc:492] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:13.668210 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d885baba6d6045e8b1ebcab6513719bb consensus state has no leader
I20260809 14:30:13.669991 31942 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet bf2fa268687d429f9fa23b182c427d6c
I20260809 14:30:13.670392 31942 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 300) took 1410 ms. Trace:
I20260809 14:30:13.672504 818 raft_consensus.cc:482] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:13.672968 818 raft_consensus.cc:504] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.675053 818 leader_election.cc:288] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:13.675985 905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Bootstrap starting.
I20260809 14:30:13.675792 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cb7842617a4779bf110e4ea254e011" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:13.670650 31942 rpcz_store.cc:276] 0809 14:30:12.259944 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.260216 (+ 272us) service_pool.cc:227] Handling call
0809 14:30:13.144018 (+883802us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9466 00007ffa4b01ab34 00007ffa4b01aac2 00007ffa4b01a9f1 00007ffa4b01a96c 00007ffa4b019c22 00007ffa492b894c 00007ffa492b7d86
0809 14:30:13.189474 (+ 45456us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:13.190541 (+ 1067us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:13.190691 (+ 150us) catalog_manager.cc:2761] Locking tablets
0809 14:30:13.191606 (+ 915us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:13.660874 (+469268us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:13.669761 (+ 8887us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:13.669894 (+ 133us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:13.670331 (+ 437us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.485482 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.486079 (+ 597us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.486119 (+ 40us) write_op.cc:435] Acquired schema lock
0809 14:30:13.486143 (+ 24us) tablet.cc:660] Decoding operations
0809 14:30:13.486632 (+ 489us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.486692 (+ 60us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.486728 (+ 36us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:13.487025 (+ 297us) tablet.cc:699] Row locks acquired
0809 14:30:13.487045 (+ 20us) write_op.cc:260] PREPARE: finished
0809 14:30:13.487217 (+ 172us) write_op.cc:270] Start()
0809 14:30:13.487344 (+ 127us) write_op.cc:276] Timestamp: P: 1786285813487135 usec, L: 0
0809 14:30:13.487372 (+ 28us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.488114 (+ 742us) log.cc:842] Serialized 564 byte log entry
0809 14:30:13.652975 (+164861us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.656258 (+ 3283us) write_op.cc:301] APPLY: starting
0809 14:30:13.656317 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.656429 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.656594 (+ 165us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.656862 (+ 268us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.657020 (+ 158us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:13.657066 (+ 46us) write_op.cc:312] APPLY: finished
0809 14:30:13.657253 (+ 187us) log.cc:842] Serialized 29 byte log entry
0809 14:30:13.657538 (+ 285us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.657609 (+ 71us) write_op.cc:454] Released schema lock
0809 14:30:13.657640 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:13.657810 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.657834 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":2029806848,"child_traces":[["op",{"apply.queue_time_us":3148,"apply.run_cpu_time_us":1585,"apply.run_wall_time_us":1720,"num_ops":2,"prepare.queue_time_us":249571,"prepare.run_cpu_time_us":3016,"prepare.run_wall_time_us":34849,"replication_time_us":165534,"threads_started":1}]]}
I20260809 14:30:13.677580 32693 tablet_bootstrap.cc:654] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.677711 32691 tablet_bootstrap.cc:492] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:13.683593 32691 tablet_bootstrap.cc:654] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.686389 847 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 221) took 1280 ms. Trace:
I20260809 14:30:13.686252 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cb7842617a4779bf110e4ea254e011" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:13.686791 847 rpcz_store.cc:276] 0809 14:30:12.405955 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.408902 (+ 2947us) service_pool.cc:227] Handling call
0809 14:30:13.311607 (+902705us) ts_tablet_manager.cc:1145] Deleting tablet 29fdaef8a95e4aa0b23018d384442d16
0809 14:30:13.311697 (+ 90us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:13.521698 (+210001us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:13.615141 (+ 93443us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:13.686303 (+ 71162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":901659}
I20260809 14:30:13.688267 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 29fdaef8a95e4aa0b23018d384442d16 (table default.Table_3c5c49afb5814baf8e9f3f33f5beac33 [id=0c17902222b34f16aab8173dbd666e86]) successfully deleted
W20260809 14:30:13.688305 32439 leader_election.cc:341] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:13.690165 32443 leader_election.cc:341] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:13.690513 32443 leader_election.cc:302] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:13.691491 818 raft_consensus.cc:2726] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:13.697942 847 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 222) took 1291 ms. Trace:
I20260809 14:30:13.699244 32705 tablet_bootstrap.cc:654] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.698320 847 rpcz_store.cc:276] 0809 14:30:12.406345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.410089 (+ 3744us) service_pool.cc:227] Handling call
0809 14:30:13.697530 (+1287441us) ts_tablet_manager.cc:1145] Deleting tablet 29cd0861b04747699ab2d13cb89b418d
0809 14:30:13.697584 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:13.697872 (+ 288us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1285610}
W20260809 14:30:13.702728 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 29cd0861b04747699ab2d13cb89b418d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29cd0861b04747699ab2d13cb89b418d already in progress: creating tablet
W20260809 14:30:13.705224 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 1d52f4b7d49544109df36c085e4aaec3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d52f4b7d49544109df36c085e4aaec3 already in progress: creating tablet
I20260809 14:30:13.721268 32705 tablet_bootstrap.cc:492] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:13.723089 32705 ts_tablet_manager.cc:1397] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.064s user 0.007s sys 0.007s
I20260809 14:30:13.727586 32705 raft_consensus.cc:348] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.728277 32705 raft_consensus.cc:374] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.734536 32705 raft_consensus.cc:729] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.740361 32705 consensus_queue.cc:260] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.749876 905 log.cc:824] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Log is configured to *not* fsync() on all Append() calls
I20260809 14:30:13.760401 32705 ts_tablet_manager.cc:1428] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.037s user 0.008s sys 0.000s
I20260809 14:30:13.763504 32705 tablet_bootstrap.cc:492] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:13.777995 32691 tablet_bootstrap.cc:492] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:13.778506 32691 ts_tablet_manager.cc:1397] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.101s user 0.007s sys 0.019s
I20260809 14:30:13.783553 32691 raft_consensus.cc:348] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.786880 32691 raft_consensus.cc:374] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.787146 32691 raft_consensus.cc:729] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.787890 32691 consensus_queue.cc:260] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.790647 32705 tablet_bootstrap.cc:654] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:13.795091 31945 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 72) took 2008 ms (client timeout 1000 ms). Trace:
I20260809 14:30:13.795153 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:13.788187 31946 catalog_manager.cc:5495] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:13.797067 843 tablet_replica.cc:330] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:13.797613 32691 ts_tablet_manager.cc:1428] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.019s user 0.007s sys 0.003s
I20260809 14:30:13.798576 32691 tablet_bootstrap.cc:492] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:13.798799 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:13.803073 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:13.804634 843 raft_consensus.cc:2229] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:13.804776 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:13.805616 843 raft_consensus.cc:2258] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:13.809003 843 ts_tablet_manager.cc:1905] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:13.806906 31946 catalog_manager.cc:5495] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:13.814131 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ee617d858c42f7b03bd287a33eebe4 with delete_type TABLET_DATA_DELETED (replaced by tablet 31f9e3b82fa94588a07af146ed78f0e4) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:13.815640 31948 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:49782 (request call id 297) took 2253 ms. Trace:
I20260809 14:30:13.819355 32705 tablet_bootstrap.cc:492] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:13.819865 32705 ts_tablet_manager.cc:1397] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.058s user 0.011s sys 0.010s
W20260809 14:30:13.819900 31939 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 946447e981224c818b5d8e0079d60559 consensus state has no leader
I20260809 14:30:13.820612 31939 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:49782 (request call id 302) took 1498 ms. Trace:
I20260809 14:30:13.821090 31939 rpcz_store.cc:276] 0809 14:30:12.322475 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.325180 (+ 2705us) service_pool.cc:227] Handling call
0809 14:30:13.141680 (+816500us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d97d6 00007ffa4b01f634 00007ffa4b01f5c2 00007ffa4b01f4f1 00007ffa4b01f46c 00007ffa4b01e722 00007ffa492b894c 00007ffa492b7d86
0809 14:30:13.220282 (+ 78602us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:13.229215 (+ 8933us) catalog_manager.cc:3616] Apply alter schema
0809 14:30:13.316516 (+ 87301us) catalog_manager.cc:3639] Acquired catalog manager lock
0809 14:30:13.316754 (+ 238us) catalog_manager.cc:3786] Updating metadata on disk
0809 14:30:13.767108 (+450354us) catalog_manager.cc:3817] Committing alterations to in-memory state
0809 14:30:13.820484 (+ 53376us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.533694 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.534621 (+ 927us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.534649 (+ 28us) write_op.cc:435] Acquired schema lock
0809 14:30:13.534700 (+ 51us) tablet.cc:660] Decoding operations
0809 14:30:13.535130 (+ 430us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.535227 (+ 97us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.535254 (+ 27us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:13.535639 (+ 385us) tablet.cc:699] Row locks acquired
0809 14:30:13.535654 (+ 15us) write_op.cc:260] PREPARE: finished
0809 14:30:13.535889 (+ 235us) write_op.cc:270] Start()
0809 14:30:13.536049 (+ 160us) write_op.cc:276] Timestamp: P: 1786285813535829 usec, L: 0
0809 14:30:13.536103 (+ 54us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.543758 (+ 7655us) log.cc:842] Serialized 405 byte log entry
0809 14:30:13.756577 (+212819us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.763533 (+ 6956us) write_op.cc:301] APPLY: starting
0809 14:30:13.763694 (+ 161us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.763947 (+ 253us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.764372 (+ 425us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.764862 (+ 490us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.765318 (+ 456us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:13.765449 (+ 131us) write_op.cc:312] APPLY: finished
0809 14:30:13.765820 (+ 371us) log.cc:842] Serialized 23 byte log entry
0809 14:30:13.766578 (+ 758us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.766728 (+ 150us) write_op.cc:454] Released schema lock
0809 14:30:13.766801 (+ 73us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:13.767541 (+ 740us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.767632 (+ 91us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":1874453760,"child_traces":[["op",{"apply.queue_time_us":6607,"apply.run_cpu_time_us":2887,"apply.run_wall_time_us":4490,"num_ops":1,"prepare.queue_time_us":215788,"prepare.run_cpu_time_us":3873,"prepare.run_wall_time_us":12873,"replication_time_us":220314}]]}
I20260809 14:30:13.831501 32705 raft_consensus.cc:348] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.832249 32705 raft_consensus.cc:374] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.832509 32705 raft_consensus.cc:729] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:13.816084 31948 rpcz_store.cc:276] 0809 14:30:11.562133 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.062095 (+499962us) service_pool.cc:227] Handling call
0809 14:30:13.145444 (+1083349us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9156 00007ffa4b016034 00007ffa4b015fc2 00007ffa4b015ef1 00007ffa4b015e6c 00007ffa4b015122 00007ffa492b894c 00007ffa492b7d86
0809 14:30:13.312177 (+166733us) catalog_manager.cc:2075] Acquired catalog manager lock
0809 14:30:13.313878 (+ 1701us) catalog_manager.cc:2121] Created new table and tablet info
0809 14:30:13.760350 (+446472us) catalog_manager.cc:2181] Wrote table and tablets to system table
0809 14:30:13.815170 (+ 54820us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0809 14:30:13.815551 (+ 381us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.523498 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.524431 (+ 933us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.524536 (+ 105us) write_op.cc:435] Acquired schema lock
0809 14:30:13.524577 (+ 41us) tablet.cc:660] Decoding operations
0809 14:30:13.525259 (+ 682us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.525344 (+ 85us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.525389 (+ 45us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:13.525807 (+ 418us) tablet.cc:699] Row locks acquired
0809 14:30:13.525849 (+ 42us) write_op.cc:260] PREPARE: finished
0809 14:30:13.526137 (+ 288us) write_op.cc:270] Start()
0809 14:30:13.526282 (+ 145us) write_op.cc:276] Timestamp: P: 1786285813526042 usec, L: 0
0809 14:30:13.526304 (+ 22us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.527390 (+ 1086us) log.cc:842] Serialized 433 byte log entry
0809 14:30:13.747937 (+220547us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.756954 (+ 9017us) write_op.cc:301] APPLY: starting
0809 14:30:13.757065 (+ 111us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.757267 (+ 202us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.757591 (+ 324us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.758632 (+ 1041us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.758893 (+ 261us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 14:30:13.758982 (+ 89us) write_op.cc:312] APPLY: finished
0809 14:30:13.759275 (+ 293us) log.cc:842] Serialized 29 byte log entry
0809 14:30:13.759973 (+ 698us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.760073 (+ 100us) write_op.cc:454] Released schema lock
0809 14:30:13.760127 (+ 54us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:13.761023 (+ 896us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.761100 (+ 77us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":2488715136,"child_traces":[["op",{"apply.queue_time_us":8724,"apply.run_cpu_time_us":3427,"apply.run_wall_time_us":4479,"num_ops":2,"prepare.queue_time_us":205476,"prepare.run_cpu_time_us":4204,"prepare.run_wall_time_us":9981,"replication_time_us":221511}]]}
I20260809 14:30:13.833104 32705 consensus_queue.cc:260] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.834235 31948 master_service.cc:946] ReplaceTablet: received request to replace tablet f06cf519a49441ae885a73e295fb585d from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:13.836658 32396 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:30:13.848393 31944 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:49782 (request call id 303) took 1240 ms. Trace:
I20260809 14:30:13.848796 31944 rpcz_store.cc:276] 0809 14:30:12.608130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.715115 (+106985us) service_pool.cc:227] Handling call
0809 14:30:13.161035 (+445920us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9156 00007ffa4b016034 00007ffa4b015fc2 00007ffa4b015ef1 00007ffa4b015e6c 00007ffa4b015122 00007ffa492b894c 00007ffa492b7d86
0809 14:30:13.314988 (+153953us) catalog_manager.cc:2075] Acquired catalog manager lock
0809 14:30:13.316226 (+ 1238us) catalog_manager.cc:2121] Created new table and tablet info
0809 14:30:13.847600 (+531374us) catalog_manager.cc:2181] Wrote table and tablets to system table
0809 14:30:13.848008 (+ 408us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0809 14:30:13.848335 (+ 327us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.666959 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.667513 (+ 554us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.667557 (+ 44us) write_op.cc:435] Acquired schema lock
0809 14:30:13.667575 (+ 18us) tablet.cc:660] Decoding operations
0809 14:30:13.667889 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.667934 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.667968 (+ 34us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:13.668182 (+ 214us) tablet.cc:699] Row locks acquired
0809 14:30:13.668198 (+ 16us) write_op.cc:260] PREPARE: finished
0809 14:30:13.668325 (+ 127us) write_op.cc:270] Start()
0809 14:30:13.668436 (+ 111us) write_op.cc:276] Timestamp: P: 1786285813668277 usec, L: 0
0809 14:30:13.668479 (+ 43us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.749893 (+ 81414us) log.cc:842] Serialized 433 byte log entry
0809 14:30:13.750895 (+ 1002us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48e471 00007ffa4b505198 00007ffa4b966ade 00007ffa4b9662b8 00007ffa4b96aba1 00007ffa4b96ab59 00007ffa4b96aae9 00007ffa4b96aab1 00007ffa4b969dad 0000000000580784 00000000005805b9
0809 14:30:13.843945 (+ 93050us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.844352 (+ 407us) write_op.cc:301] APPLY: starting
0809 14:30:13.844469 (+ 117us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.844746 (+ 277us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.844786 (+ 40us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.845627 (+ 841us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.846265 (+ 638us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 14:30:13.846355 (+ 90us) write_op.cc:312] APPLY: finished
0809 14:30:13.846698 (+ 343us) log.cc:842] Serialized 29 byte log entry
0809 14:30:13.846977 (+ 279us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.847142 (+ 165us) write_op.cc:454] Released schema lock
0809 14:30:13.847227 (+ 85us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:13.847940 (+ 713us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.847999 (+ 59us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":1023092608,"child_traces":[["op",{"apply.queue_time_us":107,"apply.run_cpu_time_us":3226,"apply.run_wall_time_us":3921,"num_ops":2,"prepare.queue_time_us":348295,"prepare.run_cpu_time_us":5656,"prepare.run_wall_time_us":84197,"raft.queue_time_us":2527,"raft.run_cpu_time_us":1242,"raft.run_wall_time_us":1242,"replication_time_us":175381,"spinlock_wait_cycles":184869248}]]}
I20260809 14:30:13.852790 31944 master_service.cc:946] ReplaceTablet: received request to replace tablet 868a8000400147c89e9d919c179fd3a0 from {username='slave'} at 127.0.0.1:49782
W20260809 14:30:13.855846 32260 consensus_peers.cc:487] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:13.858738 32166 raft_consensus.cc:3037] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:13.859898 32691 tablet_bootstrap.cc:654] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.876358 32693 tablet_bootstrap.cc:492] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:13.876876 32693 ts_tablet_manager.cc:1397] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.222s user 0.025s sys 0.026s
I20260809 14:30:13.877511 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:13.889089 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:13.889776 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_423b0a16be834083833ae85a541ff1c4 [id=c1935d3bc49c405abe4d006932820224]). Expected version 2 got 0
I20260809 14:30:13.890581 31946 catalog_manager.cc:5495] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 1 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } }
I20260809 14:30:13.891702 31946 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:13.898628 32705 ts_tablet_manager.cc:1428] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.078s user 0.001s sys 0.010s
I20260809 14:30:13.899602 32705 tablet_bootstrap.cc:492] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:13.900637 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_cc226a9bef8a4d6bb50d7bf50e57193c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:13.897646 32693 raft_consensus.cc:348] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:13.902855 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc226a9bef8a4d6bb50d7bf50e57193c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:13.901672 32693 raft_consensus.cc:374] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:13.904430 32693 raft_consensus.cc:729] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:30:13.905536 32260 consensus_peers.cc:487] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:13.905261 32693 consensus_queue.cc:260] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.919804 32693 ts_tablet_manager.cc:1428] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.043s user 0.007s sys 0.001s
I20260809 14:30:13.921537 32693 tablet_bootstrap.cc:492] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
W20260809 14:30:13.795811 31945 rpcz_store.cc:269] 0809 14:30:11.786241 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.059909 (+273668us) service_pool.cc:227] Handling call
0809 14:30:13.151412 (+1091503us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:13.794998 (+643586us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.546798 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.547796 (+ 998us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.547851 (+ 55us) write_op.cc:435] Acquired schema lock
0809 14:30:13.547863 (+ 12us) tablet.cc:660] Decoding operations
0809 14:30:13.548411 (+ 548us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.548546 (+ 135us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.548592 (+ 46us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:13.548881 (+ 289us) tablet.cc:699] Row locks acquired
0809 14:30:13.548919 (+ 38us) write_op.cc:260] PREPARE: finished
0809 14:30:13.549191 (+ 272us) write_op.cc:270] Start()
0809 14:30:13.549407 (+ 216us) write_op.cc:276] Timestamp: P: 1786285813549119 usec, L: 0
0809 14:30:13.549433 (+ 26us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.651947 (+102514us) log.cc:842] Serialized 533 byte log entry
0809 14:30:13.666689 (+ 14742us) spinlock_profiling.cc:243] Waited 98 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48e471 00007ffa4b505198 00007ffa4b966ade 00007ffa4b9662b8 00007ffa4b96aba1 00007ffa4b96ab59 00007ffa4b96aae9 00007ffa4b96aab1 00007ffa4b969dad 0000000000580784 00000000005805b9
0809 14:30:13.769276 (+102587us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.772534 (+ 3258us) write_op.cc:301] APPLY: starting
0809 14:30:13.772817 (+ 283us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.773313 (+ 496us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.773414 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.774105 (+ 691us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.775056 (+ 951us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:13.775320 (+ 264us) write_op.cc:312] APPLY: finished
0809 14:30:13.776113 (+ 793us) log.cc:842] Serialized 23 byte log entry
0809 14:30:13.777384 (+ 1271us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.777608 (+ 224us) write_op.cc:454] Released schema lock
0809 14:30:13.777704 (+ 96us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:13.778385 (+ 681us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.884297 (+105912us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":2672328832,"child_traces":[["op",{"apply.queue_time_us":2542,"apply.run_cpu_time_us":3754,"apply.run_wall_time_us":112249,"num_ops":1,"prepare.queue_time_us":228201,"prepare.run_cpu_time_us":4606,"prepare.run_wall_time_us":120066,"replication_time_us":219431,"spinlock_wait_cycles":225369472}]]}
I20260809 14:30:13.935873 32705 tablet_bootstrap.cc:654] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:13.937693 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f2894c35d51a436fb8ff7bc574624581"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:13.939752 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2894c35d51a436fb8ff7bc574624581 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:13.943041 843 ts_tablet_manager.cc:1918] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:13.944907 843 log.cc:1196] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:13.946798 843 ts_tablet_manager.cc:1939] T 07ac3ef5573c40b894c9b2e743ca0582 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:13.949215 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 07ac3ef5573c40b894c9b2e743ca0582 (table default.Table_abacee8a40094093b6e3454fedfa8ca6 [id=329d73d7b2244b42805b4b7cd0e2e45c]) successfully deleted
W20260809 14:30:13.952620 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 57ee617d858c42f7b03bd287a33eebe4 because the tablet was not found. No further retry: Not found: Tablet not found: 57ee617d858c42f7b03bd287a33eebe4
I20260809 14:30:13.962050 818 raft_consensus.cc:482] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:13.962869 818 raft_consensus.cc:504] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.969233 32169 raft_consensus.cc:1261] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Refusing update from remote peer 3eb0bc287b12421d89abf97ca66e4d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260809 14:30:13.974656 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_a4810f35181b4d12999365aa83bf91e5" } modify_external_catalogs: true
I20260809 14:30:13.975404 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_a4810f35181b4d12999365aa83bf91e5" } modify_external_catalogs: true
I20260809 14:30:13.976926 754 consensus_queue.cc:1035] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:13.985098 888 raft_consensus.cc:482] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:13.985868 888 raft_consensus.cc:504] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:13.997720 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:14.001328 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 868a8000400147c89e9d919c179fd3a0
I20260809 14:30:14.002475 32693 tablet_bootstrap.cc:654] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:14.003024 31944 catalog_manager.cc:6565] ReplaceTablet: tablet 868a8000400147c89e9d919c179fd3a0 deleted and replaced by tablet 96a0005fc1394bf9a9fde19a2d2fbada
I20260809 14:30:14.003572 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 868a8000400147c89e9d919c179fd3a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 96a0005fc1394bf9a9fde19a2d2fbada) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:14.004073 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e5e6a1c2957d4ebfaa299a5bf66454e0" } modify_external_catalogs: true
I20260809 14:30:14.008716 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 868a8000400147c89e9d919c179fd3a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 96a0005fc1394bf9a9fde19a2d2fbada) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:14.008890 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e5e6a1c2957d4ebfaa299a5bf66454e0" } modify_external_catalogs: true
I20260809 14:30:14.012125 888 leader_election.cc:288] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:14.006517 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 868a8000400147c89e9d919c179fd3a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 868a8000400147c89e9d919c179fd3a0 already in progress: creating tablet
I20260809 14:30:14.020339 754 raft_consensus.cc:482] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.020852 754 raft_consensus.cc:504] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.022922 754 leader_election.cc:288] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.025033 32396 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:30:14.024967 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:14.026162 32260 leader_election.cc:341] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:14.028036 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 868a8000400147c89e9d919c179fd3a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 868a8000400147c89e9d919c179fd3a0 already in progress: creating tablet
I20260809 14:30:14.029429 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 868a8000400147c89e9d919c179fd3a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 96a0005fc1394bf9a9fde19a2d2fbada) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:14.029863 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.030454 32515 raft_consensus.cc:2454] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:14.031066 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.032514 32260 leader_election.cc:302] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.033149 32095 leader_election.cc:302] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:14.033465 754 raft_consensus.cc:2781] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:14.033753 754 raft_consensus.cc:482] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:14.033887 935 raft_consensus.cc:2726] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:14.034034 754 raft_consensus.cc:3037] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:14.035442 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 868a8000400147c89e9d919c179fd3a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 868a8000400147c89e9d919c179fd3a0 already in progress: creating tablet
I20260809 14:30:14.027272 818 leader_election.cc:288] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:14.038190 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_32fd1fbe06e041adbd3b9aa6023f4972" } new_table_name: "default.Table_0c32f05660bc43c299491159a3fcbc45" modify_external_catalogs: true
I20260809 14:30:14.039271 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:14.039834 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:14.040422 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:14.042951 31937 catalog_manager.cc:5495] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 1 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } }
I20260809 14:30:14.044564 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:14.046242 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:14.047228 31937 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 239ad14cb3a642e08bace0759bc0b732 (table default.Table_e7dbc14f657745d390a15586fda27677 [id=284b10d1269c44dea39dc24aade3459c]). Expected version 1 got 0
I20260809 14:30:14.069161 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:14.081142 753 consensus_queue.cc:1035] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:14.102689 31939 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:49782 (request call id 304) took 1185 ms. Trace:
I20260809 14:30:14.103037 32693 tablet_bootstrap.cc:492] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:14.103529 32693 ts_tablet_manager.cc:1397] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.182s user 0.011s sys 0.000s
I20260809 14:30:14.103070 31939 rpcz_store.cc:276] 0809 14:30:12.917613 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:13.832020 (+914407us) service_pool.cc:227] Handling call
0809 14:30:13.903324 (+ 71304us) catalog_manager.cc:2075] Acquired catalog manager lock
0809 14:30:13.966267 (+ 62943us) catalog_manager.cc:2121] Created new table and tablet info
0809 14:30:14.101894 (+135627us) catalog_manager.cc:2181] Wrote table and tablets to system table
0809 14:30:14.102271 (+ 377us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0809 14:30:14.102571 (+ 300us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:14.006407 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:14.006835 (+ 428us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:14.006859 (+ 24us) write_op.cc:435] Acquired schema lock
0809 14:30:14.006877 (+ 18us) tablet.cc:660] Decoding operations
0809 14:30:14.007191 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:14.007228 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:14.007252 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:14.007480 (+ 228us) tablet.cc:699] Row locks acquired
0809 14:30:14.007493 (+ 13us) write_op.cc:260] PREPARE: finished
0809 14:30:14.007618 (+ 125us) write_op.cc:270] Start()
0809 14:30:14.007708 (+ 90us) write_op.cc:276] Timestamp: P: 1786285814007576 usec, L: 0
0809 14:30:14.007728 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 14:30:14.008262 (+ 534us) log.cc:842] Serialized 433 byte log entry
0809 14:30:14.040723 (+ 32461us) op_driver.cc:444] REPLICATION: finished
0809 14:30:14.048803 (+ 8080us) write_op.cc:301] APPLY: starting
0809 14:30:14.048942 (+ 139us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:14.049189 (+ 247us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:14.049383 (+ 194us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:14.050460 (+ 1077us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:14.050832 (+ 372us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 14:30:14.050970 (+ 138us) write_op.cc:312] APPLY: finished
0809 14:30:14.051326 (+ 356us) log.cc:842] Serialized 29 byte log entry
0809 14:30:14.051635 (+ 309us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.051767 (+ 132us) write_op.cc:454] Released schema lock
0809 14:30:14.051833 (+ 66us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":7846,"num_ops":2,"prepare.queue_time_us":110,"prepare.run_cpu_time_us":2101,"prepare.run_wall_time_us":3798,"replication_time_us":32926}]]}
I20260809 14:30:14.110801 589 raft_consensus.cc:482] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.111317 589 raft_consensus.cc:504] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.113411 589 leader_election.cc:288] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.114741 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f171260cb764ac284aef85e4219767e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:14.116680 32260 leader_election.cc:341] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.118546 941 raft_consensus.cc:482] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.118741 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f171260cb764ac284aef85e4219767e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.119563 32510 raft_consensus.cc:2454] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:14.119258 941 raft_consensus.cc:504] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.120879 32260 leader_election.cc:302] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.121054 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_60f882b13b3d48f28134afe6dfed9f10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:14.122035 589 raft_consensus.cc:2781] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:14.122339 589 raft_consensus.cc:482] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:14.122635 589 raft_consensus.cc:3037] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:14.122718 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60f882b13b3d48f28134afe6dfed9f10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:14.125097 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:14.131479 32693 raft_consensus.cc:348] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.132287 32693 raft_consensus.cc:374] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:14.140690 32693 raft_consensus.cc:729] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:14.142946 32693 consensus_queue.cc:260] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:14.147105 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 185590us
I20260809 14:30:14.150137 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88644048fa4a443fac09e951d78b3063" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:14.161074 754 raft_consensus.cc:504] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.162849 941 leader_election.cc:288] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.171049 32691 tablet_bootstrap.cc:492] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:14.171557 32691 ts_tablet_manager.cc:1397] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.373s user 0.018s sys 0.020s
I20260809 14:30:14.182773 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 24b10fe71bb44ea1ae82d0232c75daf2
I20260809 14:30:14.184957 32705 tablet_bootstrap.cc:492] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:14.184942 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b10fe71bb44ea1ae82d0232c75daf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:14 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:14.185564 32705 ts_tablet_manager.cc:1397] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.286s user 0.007s sys 0.008s
I20260809 14:30:14.187238 818 raft_consensus.cc:482] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.188004 818 raft_consensus.cc:504] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:14.188916 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 24b10fe71bb44ea1ae82d0232c75daf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 24b10fe71bb44ea1ae82d0232c75daf2 already in progress: creating tablet
I20260809 14:30:14.189066 32705 raft_consensus.cc:348] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.189687 32705 raft_consensus.cc:374] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:14.189921 32705 raft_consensus.cc:729] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:14.190716 32705 consensus_queue.cc:260] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:14.191140 32439 leader_election.cc:341] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.194281 32705 ts_tablet_manager.cc:1428] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 14:30:14.195264 32705 tablet_bootstrap.cc:492] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:14.198872 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_a5839d042df748b5b52689e11f581dd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:14.200870 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5839d042df748b5b52689e11f581dd6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:14.208880 32439 consensus_peers.cc:487] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:14.210777 941 raft_consensus.cc:482] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.211585 941 raft_consensus.cc:504] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.226488 935 raft_consensus.cc:482] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.227346 935 raft_consensus.cc:504] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.231114 935 leader_election.cc:288] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.162503 32693 ts_tablet_manager.cc:1428] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.059s user 0.007s sys 0.002s
I20260809 14:30:14.175087 589 raft_consensus.cc:504] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.196297 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b10fe71bb44ea1ae82d0232c75daf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:14 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:14.247159 32099 leader_election.cc:341] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: b0ad3946f5de4387af3627a88b6404cf
I20260809 14:30:14.247284 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f171260cb764ac284aef85e4219767e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:14.255052 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 95733a086a704e1ea397d4ffb2f2c5e7 consensus state has no leader
I20260809 14:30:14.255966 589 leader_election.cc:288] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.248198 935 raft_consensus.cc:482] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.257879 935 raft_consensus.cc:504] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:14.258955 950 log.cc:924] Time spent T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.079s user 0.001s sys 0.000s
I20260809 14:30:14.260255 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b10fe71bb44ea1ae82d0232c75daf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:14 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:14.269431 941 leader_election.cc:288] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.271268 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.272639 32510 raft_consensus.cc:2379] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:14.275460 818 leader_election.cc:288] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:14.277710 935 leader_election.cc:288] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.289773 31938 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:14.291025 31938 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:14.291636 31938 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:14.279706 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d592b1c4520467791ffd9d36e0d97fd" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:14.305569 31942 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:14.306463 31942 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:14.307212 31942 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:14.310309 31942 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:14.310810 32705 tablet_bootstrap.cc:654] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:14.311257 31942 catalog_manager.cc:5495] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:14.323608 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ac3ef5573c40b894c9b2e743ca0582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:14.329066 32439 leader_election.cc:341] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:14.325688 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 07ac3ef5573c40b894c9b2e743ca0582 because the tablet was not found. No further retry: Not found: Tablet not found: 07ac3ef5573c40b894c9b2e743ca0582
I20260809 14:30:14.331163 32095 leader_election.cc:302] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:14.335139 935 raft_consensus.cc:3037] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:14.335562 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_dad01c31cb0242f9a295c92854814711"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:14.337406 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dad01c31cb0242f9a295c92854814711 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:14.337865 31942 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_423b0a16be834083833ae85a541ff1c4 [id=c1935d3bc49c405abe4d006932820224]). Expected version 2 got 0
I20260809 14:30:14.338522 31942 catalog_manager.cc:5495] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:14.339699 31942 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
W20260809 14:30:14.342901 32095 leader_election.cc:341] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: b0ad3946f5de4387af3627a88b6404cf
I20260809 14:30:14.343330 32095 leader_election.cc:302] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:14.344134 941 raft_consensus.cc:2726] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:14.342167 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.345274 32508 raft_consensus.cc:2379] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:14.346226 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a689385d063469595fcae82aa979bb1" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.350297 32691 raft_consensus.cc:348] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.351018 32691 raft_consensus.cc:374] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:14.351302 32691 raft_consensus.cc:729] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:14.352145 32691 consensus_queue.cc:260] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:14.355825 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 24b10fe71bb44ea1ae82d0232c75daf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 24b10fe71bb44ea1ae82d0232c75daf2 already in progress: creating tablet
I20260809 14:30:14.368773 32693 tablet_bootstrap.cc:492] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:14.372430 32691 ts_tablet_manager.cc:1428] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.201s user 0.003s sys 0.005s
I20260809 14:30:14.378975 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c6a19ad0e6594d42833aa8edc7b517ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:14.385035 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6a19ad0e6594d42833aa8edc7b517ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:14.392710 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 57ee617d858c42f7b03bd287a33eebe4 with delete_type TABLET_DATA_DELETED (replaced by tablet 31f9e3b82fa94588a07af146ed78f0e4) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:14.394923 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 57ee617d858c42f7b03bd287a33eebe4 because the tablet was not found. No further retry: Not found: Tablet not found: 57ee617d858c42f7b03bd287a33eebe4
I20260809 14:30:14.375703 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:14.401445 32099 leader_election.cc:302] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:30:14.402236 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 24b10fe71bb44ea1ae82d0232c75daf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 24b10fe71bb44ea1ae82d0232c75daf2 already in progress: creating tablet
I20260809 14:30:14.402789 941 raft_consensus.cc:3037] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:14.374497 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a689385d063469595fcae82aa979bb1" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:14.409663 32099 leader_election.cc:302] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:14.410763 888 raft_consensus.cc:2726] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:14.414855 754 leader_election.cc:288] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.426054 32508 raft_consensus.cc:1261] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Refusing update from remote peer 3eb0bc287b12421d89abf97ca66e4d9b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
W20260809 14:30:14.426275 32260 leader_election.cc:341] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.426589 32508 pending_rounds.cc:76] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting all ops after (but not including) 0
I20260809 14:30:14.426959 32508 pending_rounds.cc:97] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260809 14:30:14.427418 32508 raft_consensus.cc:2866] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 14:30:14.430140 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f171260cb764ac284aef85e4219767e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
W20260809 14:30:14.430490 31938 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 73) took 1338 ms (client timeout 1000 ms). Trace:
I20260809 14:30:14.430994 32510 raft_consensus.cc:3037] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:14.431202 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b983b837948f4b0c816c275d04af0c02" } modify_external_catalogs: true
I20260809 14:30:14.432086 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b983b837948f4b0c816c275d04af0c02" } modify_external_catalogs: true
I20260809 14:30:14.435577 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:14.436409 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88644048fa4a443fac09e951d78b3063" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:14.440618 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:14.441426 32513 raft_consensus.cc:3037] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:14.421388 754 raft_consensus.cc:482] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.444307 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d592b1c4520467791ffd9d36e0d97fd" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:14.444800 754 raft_consensus.cc:504] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.455329 935 raft_consensus.cc:2726] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:14.455359 754 leader_election.cc:288] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.459021 941 raft_consensus.cc:2726] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:14.430912 31938 rpcz_store.cc:269] 0809 14:30:13.057812 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:13.470078 (+412266us) service_pool.cc:227] Handling call
0809 14:30:14.396306 (+926228us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":139,"reported_tablets":9}
I20260809 14:30:14.485157 32509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9922cad6d10945d5b013a168a3496676" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.485797 32509 raft_consensus.cc:2454] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:14.486083 955 consensus_queue.cc:1035] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:14.486351 948 consensus_queue.cc:1035] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:14.489116 32260 leader_election.cc:302] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:14.340255 31942 catalog_manager.cc:5495] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 1 to 2, leader changed from 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67) to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 2 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:14.489861 952 consensus_queue.cc:1035] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:14.490801 32510 raft_consensus.cc:2454] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:14.491652 951 raft_consensus.cc:2781] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:14.492022 951 raft_consensus.cc:482] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:14.492422 951 raft_consensus.cc:3037] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:14.490257 32260 leader_election.cc:341] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.490732 955 consensus_queue.cc:1035] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 14:30:14.499398 32260 consensus_peers.cc:487] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:14.500159 955 consensus_queue.cc:1035] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:14.501995 905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=11 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 14:30:14.503289 905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Bootstrap complete.
I20260809 14:30:14.466605 32166 raft_consensus.cc:3037] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:14.484243 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9922cad6d10945d5b013a168a3496676" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:14.517647 32260 leader_election.cc:341] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.491181 32705 tablet_bootstrap.cc:492] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:14.522883 32705 ts_tablet_manager.cc:1397] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.328s user 0.010s sys 0.008s
I20260809 14:30:14.499979 951 raft_consensus.cc:504] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.532866 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9922cad6d10945d5b013a168a3496676" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:14.533044 955 raft_consensus.cc:482] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.532166 951 leader_election.cc:288] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.533535 955 raft_consensus.cc:504] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.535836 955 leader_election.cc:288] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:14.537484 32260 leader_election.cc:341] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.539081 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bce7aec3225a4fd89b3bfd963b64582a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:14.540565 32260 leader_election.cc:341] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.541967 32705 raft_consensus.cc:348] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.542713 32705 raft_consensus.cc:374] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:14.551101 32705 raft_consensus.cc:729] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:14.554098 32705 consensus_queue.cc:260] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.565323 905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:14.573346 905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d10512ac8b3406098fad1f7da012ab3, State: Initialized, Role: FOLLOWER
I20260809 14:30:14.582428 905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 5.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:14.596246 31942 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:14.604121 905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 14:30:14.491277 32691 tablet_bootstrap.cc:492] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:30:14.507081 32443 leader_election.cc:341] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.572827 32514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9922cad6d10945d5b013a168a3496676" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:14.605350 32443 leader_election.cc:302] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.605544 32514 raft_consensus.cc:3037] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:14.606191 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 99156.3us
I20260809 14:30:14.607381 818 raft_consensus.cc:2726] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:14.607758 32443 leader_election.cc:341] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.608139 32443 leader_election.cc:302] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.606293 967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:30:14.610620 967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:30:14.611558 818 raft_consensus.cc:2726] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:14.612519 818 consensus_queue.cc:1035] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 14:30:14.592912 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bce7aec3225a4fd89b3bfd963b64582a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.629251 32514 raft_consensus.cc:2454] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:14.631865 31942 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:14.632560 32513 raft_consensus.cc:2454] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:14.610515 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_054e300a0545424c8148c2285469944e" } new_table_name: "default.Table_af577ceac58c4c31bfbaa3919493d8a2" modify_external_catalogs: true
I20260809 14:30:14.652575 32705 ts_tablet_manager.cc:1428] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.129s user 0.006s sys 0.001s
I20260809 14:30:14.676404 32705 tablet_bootstrap.cc:492] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:14.700634 589 consensus_queue.cc:1035] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 14:30:14.719907 31938 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 225029a4d923473294f4f95973f715a5 has no consensus state
I20260809 14:30:14.694573 974 tablet.cc:1722] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:14.736718 589 consensus_queue.cc:1035] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:14.738080 32691 tablet_bootstrap.cc:654] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:14.739946 818 raft_consensus.cc:482] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.740597 818 raft_consensus.cc:504] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.750164 32693 tablet_bootstrap.cc:654] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:14.751956 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:14.752908 32260 leader_election.cc:302] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:14.754243 32260 leader_election.cc:341] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.754632 32260 leader_election.cc:302] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.754783 753 raft_consensus.cc:2781] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:14.755371 951 raft_consensus.cc:2726] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:14.755452 753 raft_consensus.cc:686] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:14.756796 32260 leader_election.cc:302] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.757553 948 raft_consensus.cc:2781] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:14.758240 948 raft_consensus.cc:686] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:14.757262 753 consensus_queue.cc:237] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.762198 818 leader_election.cc:288] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:14.762900 32260 leader_election.cc:302] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.759611 948 consensus_queue.cc:237] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.764375 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74db9caa7f94059b1bba4a082fe57ef" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:14.764825 951 raft_consensus.cc:2781] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:14.765749 951 raft_consensus.cc:686] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
W20260809 14:30:14.766388 32443 leader_election.cc:341] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.766762 951 consensus_queue.cc:237] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.768555 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74db9caa7f94059b1bba4a082fe57ef" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:14.770105 32439 leader_election.cc:341] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.770466 32439 leader_election.cc:302] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:14.771178 31933 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 225029a4d923473294f4f95973f715a5 (table default.Table_af577ceac58c4c31bfbaa3919493d8a2 [id=350bc1b7fbbe4d858f15b7a8003f069f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 225029a4d923473294f4f95973f715a5 has no consensus state
I20260809 14:30:14.772606 939 raft_consensus.cc:2726] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:14.792922 32705 tablet_bootstrap.cc:654] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:14.801167 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:14.866025 32691 tablet_bootstrap.cc:492] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
W20260809 14:30:14.865944 977 log.cc:924] Time spent T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
I20260809 14:30:14.882704 948 raft_consensus.cc:482] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:14.883468 948 raft_consensus.cc:504] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.866505 32691 ts_tablet_manager.cc:1397] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.378s user 0.005s sys 0.016s
I20260809 14:30:14.886859 948 leader_election.cc:288] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.887712 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:14.888278 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:14.889148 32508 raft_consensus.cc:2454] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
W20260809 14:30:14.889878 32260 leader_election.cc:341] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.890686 32260 leader_election.cc:302] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.892282 948 raft_consensus.cc:2781] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:14.893036 948 raft_consensus.cc:482] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:14.893798 948 raft_consensus.cc:3037] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:14.894187 32691 raft_consensus.cc:348] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.894932 32691 raft_consensus.cc:374] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:14.895213 32691 raft_consensus.cc:729] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:30:14.909271 31735 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 86125.1us
I20260809 14:30:14.924207 948 raft_consensus.cc:504] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.934238 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:14.935948 32260 leader_election.cc:341] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:14.941016 948 leader_election.cc:288] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:14.941409 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:14.942024 32508 raft_consensus.cc:3037] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:14.949272 32508 raft_consensus.cc:2454] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:14.950378 32260 leader_election.cc:302] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:14.951366 948 raft_consensus.cc:2781] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:14.952054 948 raft_consensus.cc:686] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:14.952842 948 consensus_queue.cc:237] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.972656 32691 consensus_queue.cc:260] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:14.975569 32691 ts_tablet_manager.cc:1428] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.085s user 0.005s sys 0.001s
I20260809 14:30:14.976604 32691 tablet_bootstrap.cc:492] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:14.982752 32691 tablet_bootstrap.cc:654] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:14.996685 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 0bad186f0ccf4c15a4f0b16bfffa945c
I20260809 14:30:14.999737 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bad186f0ccf4c15a4f0b16bfffa945c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:13 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:15.001968 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bad186f0ccf4c15a4f0b16bfffa945c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:13 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:15.024972 32485 tablet_service.cc:1467] Processing CreateTablet for tablet 9d700191322341f998c5a8bfed609299 (DEFAULT_TABLE table=default.Table_be768f7988814d00aeb5aea5c1b5c91c [id=38c58634369648909e70d249f01651a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.027266 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d700191322341f998c5a8bfed609299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.029472 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 0d73f6b2f00e4687a8391780798f3c98 (DEFAULT_TABLE table=default.Table_6cabfb77009541cabeb67c8fd017826e [id=5d43af6fc9cd412d873da5c46ee7db30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.030462 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d73f6b2f00e4687a8391780798f3c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.044993 32486 tablet_service.cc:1467] Processing CreateTablet for tablet bb39567a82d84ebbaada271d8c2a9c9f (DEFAULT_TABLE table=default.Table_d34159d14bb64162bda06db2b81cf6c6 [id=837c72efacb3411fb4850411a57dc462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.046065 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb39567a82d84ebbaada271d8c2a9c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.055372 820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 14:30:15.058813 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 9d700191322341f998c5a8bfed609299 (DEFAULT_TABLE table=default.Table_be768f7988814d00aeb5aea5c1b5c91c [id=38c58634369648909e70d249f01651a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.060031 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 0d73f6b2f00e4687a8391780798f3c98 (DEFAULT_TABLE table=default.Table_6cabfb77009541cabeb67c8fd017826e [id=5d43af6fc9cd412d873da5c46ee7db30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.061177 32301 tablet_service.cc:1467] Processing CreateTablet for tablet bb39567a82d84ebbaada271d8c2a9c9f (DEFAULT_TABLE table=default.Table_d34159d14bb64162bda06db2b81cf6c6 [id=837c72efacb3411fb4850411a57dc462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.062862 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:15.067380 31945 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 308) took 1286 ms. Trace:
I20260809 14:30:15.068300 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d700191322341f998c5a8bfed609299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.074436 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d73f6b2f00e4687a8391780798f3c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.077036 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb39567a82d84ebbaada271d8c2a9c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.073372 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:15.079046 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:15.080075 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:15.080740 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 2a689385d063469595fcae82aa979bb1 (table default.Table_d841d100269948ca8f7fc467c85a3b0d [id=c19378f8e8994ef28e7b0eba079948c3]). Expected version 2 got 0
I20260809 14:30:15.081387 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:15.073943 31945 rpcz_store.cc:276] 0809 14:30:13.780633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:13.974316 (+193683us) service_pool.cc:227] Handling call
0809 14:30:13.975949 (+ 1633us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:13.976611 (+ 662us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:13.976764 (+ 153us) catalog_manager.cc:2761] Locking tablets
0809 14:30:14.968161 (+991397us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:14.993102 (+ 24941us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:14.993388 (+ 286us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:14.993556 (+ 168us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:15.067300 (+ 73744us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:14.970912 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:14.971339 (+ 427us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:14.971365 (+ 26us) write_op.cc:435] Acquired schema lock
0809 14:30:14.971381 (+ 16us) tablet.cc:660] Decoding operations
0809 14:30:14.971729 (+ 348us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:14.971773 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:14.971800 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:14.972101 (+ 301us) tablet.cc:699] Row locks acquired
0809 14:30:14.972121 (+ 20us) write_op.cc:260] PREPARE: finished
0809 14:30:14.972247 (+ 126us) write_op.cc:270] Start()
0809 14:30:14.972329 (+ 82us) write_op.cc:276] Timestamp: P: 1786285814972011 usec, L: 0
0809 14:30:14.972349 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 14:30:14.972922 (+ 573us) log.cc:842] Serialized 731 byte log entry
0809 14:30:14.983646 (+ 10724us) op_driver.cc:444] REPLICATION: finished
0809 14:30:14.990590 (+ 6944us) write_op.cc:301] APPLY: starting
0809 14:30:14.990673 (+ 83us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:14.990827 (+ 154us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:14.990950 (+ 123us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:14.991320 (+ 370us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:14.991537 (+ 217us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:14.991606 (+ 69us) write_op.cc:312] APPLY: finished
0809 14:30:14.991964 (+ 358us) log.cc:842] Serialized 29 byte log entry
0809 14:30:14.992202 (+ 238us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.992291 (+ 89us) write_op.cc:454] Released schema lock
0809 14:30:14.992341 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:14.992608 (+ 267us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.992647 (+ 39us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6732,"apply.run_cpu_time_us":2325,"apply.run_wall_time_us":2325,"num_ops":2,"prepare.queue_time_us":144,"prepare.run_cpu_time_us":2211,"prepare.run_wall_time_us":9681,"replication_time_us":11200}]]}
I20260809 14:30:15.103295 32693 tablet_bootstrap.cc:492] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:15.103777 32693 ts_tablet_manager.cc:1397] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.737s user 0.012s sys 0.016s
I20260809 14:30:15.107211 32693 raft_consensus.cc:348] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.108698 32693 raft_consensus.cc:374] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:15.109009 32693 raft_consensus.cc:729] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:15.109783 32693 consensus_queue.cc:260] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.115451 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 2c22055a5d334a57a6b8f2f8f9248d8b
I20260809 14:30:15.123463 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_222d7926360d490abc0717b23e7ff1f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:15.124607 32693 ts_tablet_manager.cc:1428] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.021s user 0.003s sys 0.005s
W20260809 14:30:15.126224 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_222d7926360d490abc0717b23e7ff1f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:15.138942 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c22055a5d334a57a6b8f2f8f9248d8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:14 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:15.140533 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 90e67f393f7a4bb987edc99aa86a9c90 (DEFAULT_TABLE table=default.Table_657bee986ba04a85aa02aff63c665910 [id=cab85ab775764cf8aed1e4d5d3044d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.145227 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 2c22055a5d334a57a6b8f2f8f9248d8b (DEFAULT_TABLE table=default.Table_b983b837948f4b0c816c275d04af0c02 [id=f74f7b106d224ec7bcfb46745a34684d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.148157 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_83c723c78d1b466f96b992e07866b7e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:15.149381 32693 tablet_bootstrap.cc:492] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
W20260809 14:30:15.152185 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83c723c78d1b466f96b992e07866b7e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:15.082240 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_423b0a16be834083833ae85a541ff1c4 [id=c1935d3bc49c405abe4d006932820224]). Expected version 2 got 0
I20260809 14:30:15.156843 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:15.157989 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90e67f393f7a4bb987edc99aa86a9c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.158787 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c22055a5d334a57a6b8f2f8f9248d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.175774 32693 tablet_bootstrap.cc:654] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:15.180248 31933 catalog_manager.cc:4821] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 0bad186f0ccf4c15a4f0b16bfffa945c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bad186f0ccf4c15a4f0b16bfffa945c already in progress: creating tablet
I20260809 14:30:15.191756 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:15.193881 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:15.194813 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:15.195586 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:15.196898 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:15.209357 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_d4baddb55ef84a0f9456db8bb8782728"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:15.211692 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4baddb55ef84a0f9456db8bb8782728 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:15.215536 32489 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c22055a5d334a57a6b8f2f8f9248d8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:14 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:15.220763 32693 tablet_bootstrap.cc:492] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:15.222693 32693 ts_tablet_manager.cc:1397] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.073s user 0.008s sys 0.007s
I20260809 14:30:15.216742 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 9121614514574d0594b1d5fe89fcc366 (DEFAULT_TABLE table=default.Table_de805f1551304f6a92cf7d4db934dcda [id=653ab7bb85544b38a370d0a7b61a9714]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.231606 32141 tablet_service.cc:1467] Processing CreateTablet for tablet 9d700191322341f998c5a8bfed609299 (DEFAULT_TABLE table=default.Table_be768f7988814d00aeb5aea5c1b5c91c [id=38c58634369648909e70d249f01651a6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:15.232725 984 log.cc:924] Time spent T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.310s user 0.006s sys 0.000s
I20260809 14:30:15.232914 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d700191322341f998c5a8bfed609299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.233332 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 0d73f6b2f00e4687a8391780798f3c98 (DEFAULT_TABLE table=default.Table_6cabfb77009541cabeb67c8fd017826e [id=5d43af6fc9cd412d873da5c46ee7db30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.235715 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 90e67f393f7a4bb987edc99aa86a9c90 (DEFAULT_TABLE table=default.Table_657bee986ba04a85aa02aff63c665910 [id=cab85ab775764cf8aed1e4d5d3044d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.236052 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 1a42b931475a407c971911e55d562d70 (DEFAULT_TABLE table=default.Table_513f49ccff6d4f15bd52a9814a93e513 [id=ff2ac23bc7f74a978e7bfad54124ffea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.236819 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90e67f393f7a4bb987edc99aa86a9c90. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:15.253350 31933 catalog_manager.cc:4808] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): delete failed for tablet 2c22055a5d334a57a6b8f2f8f9248d8b because the tablet was not found. No further retry: Not found: Tablet not found: 2c22055a5d334a57a6b8f2f8f9248d8b
I20260809 14:30:15.257264 32224 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:30:15.257815 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:15.258530 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 90e67f393f7a4bb987edc99aa86a9c90 (DEFAULT_TABLE table=default.Table_657bee986ba04a85aa02aff63c665910 [id=cab85ab775764cf8aed1e4d5d3044d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.259835 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90e67f393f7a4bb987edc99aa86a9c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.261768 32481 tablet_service.cc:1467] Processing CreateTablet for tablet 9121614514574d0594b1d5fe89fcc366 (DEFAULT_TABLE table=default.Table_de805f1551304f6a92cf7d4db934dcda [id=653ab7bb85544b38a370d0a7b61a9714]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.263170 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9121614514574d0594b1d5fe89fcc366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.264739 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d73f6b2f00e4687a8391780798f3c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.268539 32143 tablet_service.cc:1467] Processing CreateTablet for tablet bb39567a82d84ebbaada271d8c2a9c9f (DEFAULT_TABLE table=default.Table_d34159d14bb64162bda06db2b81cf6c6 [id=837c72efacb3411fb4850411a57dc462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.268859 32478 tablet_service.cc:1467] Processing CreateTablet for tablet 2c22055a5d334a57a6b8f2f8f9248d8b (DEFAULT_TABLE table=default.Table_b983b837948f4b0c816c275d04af0c02 [id=f74f7b106d224ec7bcfb46745a34684d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.269608 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb39567a82d84ebbaada271d8c2a9c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.270226 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c22055a5d334a57a6b8f2f8f9248d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.274344 32136 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bad186f0ccf4c15a4f0b16bfffa945c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:13 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:15.299197 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:15.300441 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 88644048fa4a443fac09e951d78b3063 (table default.Table_569de184b14243abb58d1414941be5aa [id=67767d6db83f4ae6ab265b854d2ad695]). Expected version 2 got 0
I20260809 14:30:15.301141 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 239ad14cb3a642e08bace0759bc0b732 (table default.Table_e7dbc14f657745d390a15586fda27677 [id=284b10d1269c44dea39dc24aade3459c]). Expected version 1 got 0
I20260809 14:30:15.301756 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:15.230572 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9121614514574d0594b1d5fe89fcc366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.315485 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:15.323819 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:15.326872 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:15.226706 32693 raft_consensus.cc:348] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.333855 32693 raft_consensus.cc:374] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:15.334221 32693 raft_consensus.cc:729] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:15.334901 32693 consensus_queue.cc:260] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.343845 32308 tablet_service.cc:1467] Processing CreateTablet for tablet c4f4c1eb916c4b51a5e176d541da8574 (DEFAULT_TABLE table=default.Table_6cb89cdda08047ada14df086bf662194 [id=a752eb35686e4e13969893a14d548da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.345345 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4f4c1eb916c4b51a5e176d541da8574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.347571 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 1a42b931475a407c971911e55d562d70 (DEFAULT_TABLE table=default.Table_513f49ccff6d4f15bd52a9814a93e513 [id=ff2ac23bc7f74a978e7bfad54124ffea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.349982 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a42b931475a407c971911e55d562d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.358026 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 1a42b931475a407c971911e55d562d70 (DEFAULT_TABLE table=default.Table_513f49ccff6d4f15bd52a9814a93e513 [id=ff2ac23bc7f74a978e7bfad54124ffea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.362844 32693 ts_tablet_manager.cc:1428] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.139s user 0.006s sys 0.000s
I20260809 14:30:15.364086 32693 tablet_bootstrap.cc:492] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:15.376937 32479 tablet_service.cc:1467] Processing CreateTablet for tablet c4f4c1eb916c4b51a5e176d541da8574 (DEFAULT_TABLE table=default.Table_6cb89cdda08047ada14df086bf662194 [id=a752eb35686e4e13969893a14d548da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.381453 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 2af4e169f518447c8d29c630dca93ff9 (DEFAULT_TABLE table=default.Table_ed4114d732c44266bc8e100a01ce3499 [id=b5a42e2ab9594bba8f9ce9a39d67532e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.382498 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af4e169f518447c8d29c630dca93ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.386093 31939 catalog_manager.cc:5495] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 1 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } }
I20260809 14:30:15.387907 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:15.393137 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a42b931475a407c971911e55d562d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.395135 32145 tablet_service.cc:1467] Processing CreateTablet for tablet c4f4c1eb916c4b51a5e176d541da8574 (DEFAULT_TABLE table=default.Table_6cb89cdda08047ada14df086bf662194 [id=a752eb35686e4e13969893a14d548da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.396431 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4f4c1eb916c4b51a5e176d541da8574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.400081 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 9121614514574d0594b1d5fe89fcc366 (DEFAULT_TABLE table=default.Table_de805f1551304f6a92cf7d4db934dcda [id=653ab7bb85544b38a370d0a7b61a9714]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.401291 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9121614514574d0594b1d5fe89fcc366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.391932 935 raft_consensus.cc:482] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:15.464720 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c22055a5d334a57a6b8f2f8f9248d8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:14 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:15.478639 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 2af4e169f518447c8d29c630dca93ff9 (DEFAULT_TABLE table=default.Table_ed4114d732c44266bc8e100a01ce3499 [id=b5a42e2ab9594bba8f9ce9a39d67532e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:15.466012 984 log.cc:924] Time spent T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.231s user 0.003s sys 0.000s
I20260809 14:30:15.485528 948 raft_consensus.cc:482] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:15.486217 948 raft_consensus.cc:504] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.490818 948 leader_election.cc:288] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:15.465672 935 raft_consensus.cc:504] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.493371 935 leader_election.cc:288] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:15.499845 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a42b931475a407c971911e55d562d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.500391 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 99b4c2a9dce646dda38647ef011b8260 (DEFAULT_TABLE table=default.Table_ca71a97e95374666b5d79f35075e26b3 [id=becaeb80c3214075b827d7e730867168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.501973 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b4c2a9dce646dda38647ef011b8260. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:15.505892 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 0bad186f0ccf4c15a4f0b16bfffa945c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bad186f0ccf4c15a4f0b16bfffa945c already in progress: creating tablet
I20260809 14:30:15.507041 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4f4c1eb916c4b51a5e176d541da8574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.512718 32705 tablet_bootstrap.cc:492] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:15.513100 32705 ts_tablet_manager.cc:1397] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.837s user 0.016s sys 0.007s
W20260809 14:30:15.515506 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 2c22055a5d334a57a6b8f2f8f9248d8b because the tablet was not found. No further retry: Not found: Tablet not found: 2c22055a5d334a57a6b8f2f8f9248d8b
I20260809 14:30:15.518023 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 99b4c2a9dce646dda38647ef011b8260 (DEFAULT_TABLE table=default.Table_ca71a97e95374666b5d79f35075e26b3 [id=becaeb80c3214075b827d7e730867168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.519121 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b4c2a9dce646dda38647ef011b8260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.519179 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 225029a4d923473294f4f95973f715a5 (DEFAULT_TABLE table=default.Table_af577ceac58c4c31bfbaa3919493d8a2 [id=350bc1b7fbbe4d858f15b7a8003f069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.519263 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d592b1c4520467791ffd9d36e0d97fd" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:15.520332 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225029a4d923473294f4f95973f715a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.520155 32508 raft_consensus.cc:2454] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:15.525244 32260 leader_election.cc:302] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:15.538751 31944 master_service.cc:946] ReplaceTablet: received request to replace tablet 7856076c1332479a894d2d69d16b19f5 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:15.540997 31944 master_service.cc:946] ReplaceTablet: received request to replace tablet bce7aec3225a4fd89b3bfd963b64582a from {username='slave'} at 127.0.0.1:49782
W20260809 14:30:15.542666 32691 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.557s user 0.117s sys 0.216s
W20260809 14:30:15.542979 32691 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.558s user 0.117s sys 0.216s
I20260809 14:30:15.545637 32691 tablet_bootstrap.cc:492] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:15.546061 32691 ts_tablet_manager.cc:1397] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.570s user 0.128s sys 0.216s
I20260809 14:30:15.548055 1007 raft_consensus.cc:2781] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:15.548712 1007 raft_consensus.cc:482] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:15.549402 32693 tablet_bootstrap.cc:654] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:15.550096 32691 raft_consensus.cc:348] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.550777 32691 raft_consensus.cc:374] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:15.551047 32691 raft_consensus.cc:729] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:15.551716 32691 consensus_queue.cc:260] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.554111 32691 ts_tablet_manager.cc:1428] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260809 14:30:15.557966 32705 raft_consensus.cc:348] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.558720 32705 raft_consensus.cc:374] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:15.556408 32477 tablet_service.cc:1467] Processing CreateTablet for tablet 225029a4d923473294f4f95973f715a5 (DEFAULT_TABLE table=default.Table_af577ceac58c4c31bfbaa3919493d8a2 [id=350bc1b7fbbe4d858f15b7a8003f069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.558996 32705 raft_consensus.cc:729] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:15.560105 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225029a4d923473294f4f95973f715a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.549329 1007 raft_consensus.cc:3037] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:15.570817 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 31f9e3b82fa94588a07af146ed78f0e4 (DEFAULT_TABLE table=default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.571774 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31f9e3b82fa94588a07af146ed78f0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.573882 32691 tablet_bootstrap.cc:492] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:15.578573 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_971214cd6b5048afb1c306f0b0c55219"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:15.580596 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_971214cd6b5048afb1c306f0b0c55219 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:15.389357 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 32cb7842617a4779bf110e4ea254e011 (table default.Table_b86f8f5a1acc4bacb6b93b4acd413b0f [id=2d493ef1e83d47da9117ecd5312dacdc]). Expected version 1 got 0
I20260809 14:30:15.591387 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 239ad14cb3a642e08bace0759bc0b732 (table default.Table_e7dbc14f657745d390a15586fda27677 [id=284b10d1269c44dea39dc24aade3459c]). Expected version 1 got 0
I20260809 14:30:15.592214 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 88644048fa4a443fac09e951d78b3063 (table default.Table_569de184b14243abb58d1414941be5aa [id=67767d6db83f4ae6ab265b854d2ad695]). Expected version 2 got 0
I20260809 14:30:15.593386 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
W20260809 14:30:15.594653 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 69473.8us
I20260809 14:30:15.600517 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555 (DEFAULT_TABLE table=default.Table_b46cfe02c4084ffbb8825b06376fc849 [id=7f0923619d004dcbb14d481d42ca3341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.602366 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.624565 939 raft_consensus.cc:482] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:15.625064 939 raft_consensus.cc:504] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.626920 939 leader_election.cc:288] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:15.626941 32691 tablet_bootstrap.cc:654] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:15.650645 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af4e169f518447c8d29c630dca93ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.656067 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:15.658566 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet e74db9caa7f94059b1bba4a082fe57ef (table default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]). Expected version 1 got 0
I20260809 14:30:15.662546 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:15.688534 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet bce7aec3225a4fd89b3bfd963b64582a
I20260809 14:30:15.689082 31945 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:49782 (request call id 320) took 1052 ms. Trace:
I20260809 14:30:15.690389 31944 catalog_manager.cc:6565] ReplaceTablet: tablet bce7aec3225a4fd89b3bfd963b64582a deleted and replaced by tablet 408ee42ea5844741b9ab514cbe6a94e4
W20260809 14:30:15.690434 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 0bad186f0ccf4c15a4f0b16bfffa945c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bad186f0ccf4c15a4f0b16bfffa945c already in progress: creating tablet
W20260809 14:30:15.691951 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 2c22055a5d334a57a6b8f2f8f9248d8b because the tablet was not found. No further retry: Not found: Tablet not found: 2c22055a5d334a57a6b8f2f8f9248d8b
I20260809 14:30:15.707893 32489 tablet_service.cc:1514] Processing DeleteTablet for tablet bce7aec3225a4fd89b3bfd963b64582a with delete_type TABLET_DATA_DELETED (replaced by tablet 408ee42ea5844741b9ab514cbe6a94e4) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:15.709879 32131 tablet_service.cc:1514] Processing DeleteTablet for tablet bce7aec3225a4fd89b3bfd963b64582a with delete_type TABLET_DATA_DELETED (replaced by tablet 408ee42ea5844741b9ab514cbe6a94e4) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:15.714879 32134 tablet_service.cc:1467] Processing CreateTablet for tablet 225029a4d923473294f4f95973f715a5 (DEFAULT_TABLE table=default.Table_af577ceac58c4c31bfbaa3919493d8a2 [id=350bc1b7fbbe4d858f15b7a8003f069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.719556 32133 tablet_service.cc:1467] Processing CreateTablet for tablet 31f9e3b82fa94588a07af146ed78f0e4 (DEFAULT_TABLE table=default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.725981 32132 tablet_service.cc:1467] Processing CreateTablet for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555 (DEFAULT_TABLE table=default.Table_b46cfe02c4084ffbb8825b06376fc849 [id=7f0923619d004dcbb14d481d42ca3341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.732517 32134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225029a4d923473294f4f95973f715a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.744328 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:15.745270 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:15.745831 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:15.746949 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:15.747678 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:15.748430 31938 catalog_manager.cc:5495] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:15.750326 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:15.758206 31938 catalog_manager.cc:5495] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:15.760672 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_423b0a16be834083833ae85a541ff1c4 [id=c1935d3bc49c405abe4d006932820224]). Expected version 2 got 0
I20260809 14:30:15.761662 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 2c22055a5d334a57a6b8f2f8f9248d8b (DEFAULT_TABLE table=default.Table_b983b837948f4b0c816c275d04af0c02 [id=f74f7b106d224ec7bcfb46745a34684d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.762418 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:15.765246 996 tablet_replica.cc:330] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:15.766976 996 raft_consensus.cc:2229] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:15.767673 996 raft_consensus.cc:2258] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:15.769568 996 ts_tablet_manager.cc:1905] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:15.773952 32095 leader_election.cc:341] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 29fdaef8a95e4aa0b23018d384442d16
I20260809 14:30:15.781679 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 2af4e169f518447c8d29c630dca93ff9 (DEFAULT_TABLE table=default.Table_ed4114d732c44266bc8e100a01ce3499 [id=b5a42e2ab9594bba8f9ce9a39d67532e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.764111 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 239ad14cb3a642e08bace0759bc0b732 (table default.Table_e7dbc14f657745d390a15586fda27677 [id=284b10d1269c44dea39dc24aade3459c]). Expected version 1 got 0
I20260809 14:30:15.796335 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 99b4c2a9dce646dda38647ef011b8260 (DEFAULT_TABLE table=default.Table_ca71a97e95374666b5d79f35075e26b3 [id=becaeb80c3214075b827d7e730867168]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:15.806422 31933 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a0b196e429514cbc803e9a99a986d4d2 (table default.Table_42a687657ecc485a9c3d956e245784c5 [id=58e17b048a594020a069730291d81345], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a0b196e429514cbc803e9a99a986d4d2 consensus state has no leader
I20260809 14:30:15.829643 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:15.830564 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:15.831519 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:15.831986 32691 tablet_bootstrap.cc:492] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:15.832372 32691 ts_tablet_manager.cc:1397] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.259s user 0.015s sys 0.027s
I20260809 14:30:15.834600 32691 raft_consensus.cc:348] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.835273 32691 raft_consensus.cc:374] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:15.835530 32691 raft_consensus.cc:729] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:15.689386 31945 rpcz_store.cc:276] 0809 14:30:14.636340 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.498902 (+862562us) service_pool.cc:227] Handling call
0809 14:30:15.578400 (+ 79498us) spinlock_profiling.cc:243] Waited 78.4 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9156 00007ffa4b016034 00007ffa4b015fc2 00007ffa4b015ef1 00007ffa4b015e6c 00007ffa4b015122 00007ffa492b894c 00007ffa492b7d86
0809 14:30:15.644326 (+ 65926us) catalog_manager.cc:2075] Acquired catalog manager lock
0809 14:30:15.645795 (+ 1469us) catalog_manager.cc:2121] Created new table and tablet info
0809 14:30:15.688410 (+ 42615us) catalog_manager.cc:2181] Wrote table and tablets to system table
0809 14:30:15.688744 (+ 334us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0809 14:30:15.689028 (+ 284us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:15.648648 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:15.649189 (+ 541us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:15.649218 (+ 29us) write_op.cc:435] Acquired schema lock
0809 14:30:15.649233 (+ 15us) tablet.cc:660] Decoding operations
0809 14:30:15.649566 (+ 333us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:15.649612 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:15.649638 (+ 26us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:15.649922 (+ 284us) tablet.cc:699] Row locks acquired
0809 14:30:15.649946 (+ 24us) write_op.cc:260] PREPARE: finished
0809 14:30:15.650082 (+ 136us) write_op.cc:270] Start()
0809 14:30:15.650168 (+ 86us) write_op.cc:276] Timestamp: P: 1786285815650031 usec, L: 0
0809 14:30:15.650187 (+ 19us) op_driver.cc:331] REPLICATION: starting
0809 14:30:15.650739 (+ 552us) log.cc:842] Serialized 433 byte log entry
0809 14:30:15.665898 (+ 15159us) op_driver.cc:444] REPLICATION: finished
0809 14:30:15.674894 (+ 8996us) write_op.cc:301] APPLY: starting
0809 14:30:15.674952 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:15.675064 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:15.675170 (+ 106us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:15.675615 (+ 445us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:15.675762 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 14:30:15.675808 (+ 46us) write_op.cc:312] APPLY: finished
0809 14:30:15.676005 (+ 197us) log.cc:842] Serialized 29 byte log entry
0809 14:30:15.676677 (+ 672us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:15.676750 (+ 73us) write_op.cc:454] Released schema lock
0809 14:30:15.676782 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:15.676959 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:15.676987 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":180298624,"child_traces":[["op",{"apply.queue_time_us":8841,"apply.run_cpu_time_us":1755,"apply.run_wall_time_us":2272,"num_ops":2,"prepare.queue_time_us":1736,"prepare.run_cpu_time_us":2445,"prepare.run_wall_time_us":8693,"raft.queue_time_us":118,"raft.run_cpu_time_us":680,"raft.run_wall_time_us":1477,"replication_time_us":15642}]]}
I20260809 14:30:15.843518 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 31f9e3b82fa94588a07af146ed78f0e4 (DEFAULT_TABLE table=default.Table_710d9b1281b94c84a497faea0c441199 [id=d32aa28ad50644ec87bca1453f9844b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.844846 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31f9e3b82fa94588a07af146ed78f0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.849802 32691 consensus_queue.cc:260] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.850657 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555 (DEFAULT_TABLE table=default.Table_b46cfe02c4084ffbb8825b06376fc849 [id=7f0923619d004dcbb14d481d42ca3341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:15.851887 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.559753 32705 consensus_queue.cc:260] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.858368 32296 tablet_service.cc:1514] Processing DeleteTablet for tablet bce7aec3225a4fd89b3bfd963b64582a with delete_type TABLET_DATA_DELETED (replaced by tablet 408ee42ea5844741b9ab514cbe6a94e4) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:15.838903 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:15.859098 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:15.859716 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:15.860757 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_423b0a16be834083833ae85a541ff1c4 [id=c1935d3bc49c405abe4d006932820224]). Expected version 2 got 0
I20260809 14:30:15.861548 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 88644048fa4a443fac09e951d78b3063 (table default.Table_569de184b14243abb58d1414941be5aa [id=67767d6db83f4ae6ab265b854d2ad695]). Expected version 2 got 0
I20260809 14:30:15.862111 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:15.862572 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet e74db9caa7f94059b1bba4a082fe57ef (table default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]). Expected version 1 got 0
I20260809 14:30:15.866775 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:15.868273 32691 ts_tablet_manager.cc:1428] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.036s user 0.004s sys 0.001s
I20260809 14:30:15.870647 32705 ts_tablet_manager.cc:1428] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.357s user 0.007s sys 0.000s
I20260809 14:30:15.872042 32166 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.30.252.66:52479 (request call id 138) took 1073 ms. Trace:
I20260809 14:30:15.872550 32166 rpcz_store.cc:276] 0809 14:30:14.798055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:14.798756 (+ 701us) service_pool.cc:227] Handling call
0809 14:30:14.799390 (+ 634us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 14:30:14.799984 (+ 594us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 14:30:14.800050 (+ 66us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 14:30:14.802756 (+ 2706us) log.cc:842] Serialized 26 byte log entry
0809 14:30:14.908394 (+105638us) raft_consensus.cc:1631] Marking committed up to 1
0809 14:30:14.915024 (+ 6630us) log.cc:842] Serialized 14 byte log entry
0809 14:30:14.917038 (+ 2014us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 14:30:14.917574 (+ 536us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 14:30:15.231749 (+314175us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b4400037e0c. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c00ebf11 00007fb6c01b9a95 00007fb6c01ba4b2 00007fb6c7024511 00007fb6c701fe1a 00007fb6c7013bee 00007fb6c7012f36 00007fb6c7012b58 00007fb6c701c551 00007fb6c701c509 00007fb6c701c499
0809 14:30:15.871527 (+639778us) raft_consensus.cc:1671] finished
0809 14:30:15.871655 (+ 128us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 14:30:15.871841 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":399,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":26,"raft.queue_time_us":9195,"raft.run_cpu_time_us":432,"raft.run_wall_time_us":76662,"spinlock_wait_cycles":6014720,"thread_start_us":105131,"threads_started":1,"wal-append.queue_time_us":118455}
I20260809 14:30:15.879712 32691 tablet_bootstrap.cc:492] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:15.880728 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 32cb7842617a4779bf110e4ea254e011 (table default.Table_b86f8f5a1acc4bacb6b93b4acd413b0f [id=2d493ef1e83d47da9117ecd5312dacdc]). Expected version 1 got 0
I20260809 14:30:15.881314 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 239ad14cb3a642e08bace0759bc0b732 (table default.Table_e7dbc14f657745d390a15586fda27677 [id=284b10d1269c44dea39dc24aade3459c]). Expected version 1 got 0
I20260809 14:30:15.883556 996 ts_tablet_manager.cc:1918] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:15.883909 996 log.cc:1196] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/bce7aec3225a4fd89b3bfd963b64582a
I20260809 14:30:15.884559 996 ts_tablet_manager.cc:1939] T bce7aec3225a4fd89b3bfd963b64582a P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:15.884684 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac7afd531a746cab7618ead98b4c4d9" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:15.881911 1007 raft_consensus.cc:504] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.885754 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet bce7aec3225a4fd89b3bfd963b64582a (table default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e]) successfully deleted
I20260809 14:30:15.886626 1007 leader_election.cc:288] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:15.896061 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:15.898821 32691 tablet_bootstrap.cc:654] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:15.913370 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 1eb9d7c27381473aa87b718cca57b678) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:15.915019 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 29fdaef8a95e4aa0b23018d384442d16 because the tablet was not found. No further retry: Not found: Tablet not found: 29fdaef8a95e4aa0b23018d384442d16
I20260809 14:30:15.917613 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 with delete_type TABLET_DATA_DELETED (replaced by tablet b393017627934fa1b740d8ba8ac62d51) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:15.918114 31937 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 46) took 3555 ms (client timeout 1000 ms). Trace:
I20260809 14:30:15.918848 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:15.919509 996 tablet_replica.cc:330] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:15.920334 996 raft_consensus.cc:2229] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:15.920868 996 raft_consensus.cc:2258] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:15.904151 31946 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_d841d100269948ca8f7fc467c85a3b0d [id=c19378f8e8994ef28e7b0eba079948c3] alter complete (version 2)
I20260809 14:30:15.927057 996 ts_tablet_manager.cc:1905] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:15.930971 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 1eb9d7c27381473aa87b718cca57b678) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:15.933050 31946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 56) took 3547 ms (client timeout 1000 ms). Trace:
W20260809 14:30:15.937104 31948 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 74) took 1713 ms (client timeout 1000 ms). Trace:
W20260809 14:30:15.937407 31948 rpcz_store.cc:269] 0809 14:30:14.223758 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:14.270854 (+ 47096us) service_pool.cc:227] Handling call
0809 14:30:14.317329 (+ 46475us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:15.937042 (+1619713us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18343,"reported_tablets":12,"spinlock_wait_cycles":73153920}
I20260809 14:30:15.946355 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad3946f5de4387af3627a88b6404cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:15.949812 32163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac7afd531a746cab7618ead98b4c4d9" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:15.953840 32439 leader_election.cc:341] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:15.933379 31946 rpcz_store.cc:269] 0809 14:30:12.385216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.386591 (+ 1375us) service_pool.cc:227] Handling call
0809 14:30:13.157285 (+770694us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:15.932986 (+2775701us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.969499 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.969986 (+ 487us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.970013 (+ 27us) write_op.cc:435] Acquired schema lock
0809 14:30:13.970034 (+ 21us) tablet.cc:660] Decoding operations
0809 14:30:13.970676 (+ 642us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.970729 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.970763 (+ 34us) tablet.cc:683] Acquiring locks for 3 operations
0809 14:30:13.971161 (+ 398us) tablet.cc:699] Row locks acquired
0809 14:30:13.971185 (+ 24us) write_op.cc:260] PREPARE: finished
0809 14:30:13.971336 (+ 151us) write_op.cc:270] Start()
0809 14:30:13.971450 (+ 114us) write_op.cc:276] Timestamp: P: 1786285813971269 usec, L: 0
0809 14:30:13.971476 (+ 26us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.972180 (+ 704us) log.cc:842] Serialized 1293 byte log entry
0809 14:30:13.989262 (+ 17082us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.991179 (+ 1917us) write_op.cc:301] APPLY: starting
0809 14:30:13.991255 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.991495 (+ 240us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.991741 (+ 246us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.992555 (+ 814us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.992869 (+ 314us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0809 14:30:13.992989 (+ 120us) write_op.cc:312] APPLY: finished
0809 14:30:13.993340 (+ 351us) log.cc:842] Serialized 35 byte log entry
0809 14:30:13.993578 (+ 238us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.993696 (+ 118us) write_op.cc:454] Released schema lock
0809 14:30:13.993785 (+ 89us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:14.036601 (+ 42816us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.036632 (+ 31us) write_op.cc:454] Released schema lock
Related trace 'op':
0809 14:30:14.042467 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:14.042997 (+ 530us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:14.043103 (+ 106us) write_op.cc:435] Acquired schema lock
0809 14:30:14.043131 (+ 28us) tablet.cc:660] Decoding operations
0809 14:30:14.077848 (+ 34717us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:14.077938 (+ 90us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:14.077974 (+ 36us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:14.078317 (+ 343us) tablet.cc:699] Row locks acquired
0809 14:30:14.078344 (+ 27us) write_op.cc:260] PREPARE: finished
0809 14:30:14.078575 (+ 231us) write_op.cc:270] Start()
0809 14:30:14.078754 (+ 179us) write_op.cc:276] Timestamp: P: 1786285814078507 usec, L: 0
0809 14:30:14.078774 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 14:30:14.080192 (+ 1418us) log.cc:842] Serialized 356 byte log entry
0809 14:30:14.096739 (+ 16547us) op_driver.cc:444] REPLICATION: finished
0809 14:30:14.099296 (+ 2557us) write_op.cc:301] APPLY: starting
0809 14:30:14.099375 (+ 79us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:14.099537 (+ 162us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:14.099708 (+ 171us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:14.100059 (+ 351us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:14.100378 (+ 319us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:14.100490 (+ 112us) write_op.cc:312] APPLY: finished
0809 14:30:14.100777 (+ 287us) log.cc:842] Serialized 23 byte log entry
0809 14:30:14.105908 (+ 5131us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.106299 (+ 391us) write_op.cc:454] Released schema lock
0809 14:30:14.106374 (+ 75us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:14.106629 (+ 255us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.106683 (+ 54us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16786,"reported_tablets":10,"spinlock_wait_cycles":1763598592,"child_traces":[["op",{"apply.queue_time_us":1711,"apply.run_cpu_time_us":3168,"apply.run_wall_time_us":45670,"num_ops":3,"prepare.queue_time_us":87,"prepare.run_cpu_time_us":3064,"prepare.run_wall_time_us":4518,"raft.queue_time_us":116,"raft.run_cpu_time_us":872,"raft.run_wall_time_us":872,"replication_time_us":17695,"spinlock_wait_cycles":231424}],["op",{"apply.queue_time_us":2346,"apply.run_cpu_time_us":2461,"apply.run_wall_time_us":7581,"num_ops":1,"prepare.queue_time_us":126,"prepare.run_cpu_time_us":4238,"prepare.run_wall_time_us":40090,"raft.queue_time_us":1326,"raft.run_cpu_time_us":1363,"raft.run_wall_time_us":1388,"replication_time_us":17862}]]}
W20260809 14:30:15.956013 31942 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 57) took 2560 ms (client timeout 1000 ms). Trace:
I20260809 14:30:15.957245 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d592b1c4520467791ffd9d36e0d97fd" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:15.956382 31942 rpcz_store.cc:269] 0809 14:30:13.395435 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:13.679073 (+283638us) service_pool.cc:227] Handling call
0809 14:30:13.752825 (+ 73752us) spinlock_profiling.cc:243] Waited 60.8 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:14.663553 (+910728us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.663591 (+ 38us) write_op.cc:454] Released schema lock
0809 14:30:15.955938 (+1292347us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:14.634597 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:14.635459 (+ 862us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:14.635498 (+ 39us) write_op.cc:435] Acquired schema lock
0809 14:30:14.635523 (+ 25us) tablet.cc:660] Decoding operations
0809 14:30:14.636427 (+ 904us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:14.636566 (+ 139us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:14.636605 (+ 39us) tablet.cc:683] Acquiring locks for 3 operations
0809 14:30:14.637387 (+ 782us) tablet.cc:699] Row locks acquired
0809 14:30:14.637529 (+ 142us) write_op.cc:260] PREPARE: finished
0809 14:30:14.637763 (+ 234us) write_op.cc:270] Start()
0809 14:30:14.637960 (+ 197us) write_op.cc:276] Timestamp: P: 1786285814637525 usec, L: 0
0809 14:30:14.638020 (+ 60us) op_driver.cc:331] REPLICATION: starting
0809 14:30:14.640100 (+ 2080us) log.cc:842] Serialized 1293 byte log entry
0809 14:30:14.655794 (+ 15694us) op_driver.cc:444] REPLICATION: finished
0809 14:30:14.656091 (+ 297us) write_op.cc:301] APPLY: starting
0809 14:30:14.656161 (+ 70us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:14.656311 (+ 150us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:14.656448 (+ 137us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:14.657228 (+ 780us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:14.657469 (+ 241us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0809 14:30:14.657545 (+ 76us) write_op.cc:312] APPLY: finished
0809 14:30:14.657855 (+ 310us) log.cc:842] Serialized 35 byte log entry
0809 14:30:14.658057 (+ 202us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.658174 (+ 117us) write_op.cc:454] Released schema lock
0809 14:30:14.658247 (+ 73us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":28986,"reported_tablets":16,"spinlock_wait_cycles":139746688,"child_traces":[["op",{"apply.queue_time_us":96,"apply.run_cpu_time_us":2466,"apply.run_wall_time_us":2466,"num_ops":3,"prepare.queue_time_us":89,"prepare.run_cpu_time_us":5215,"prepare.run_wall_time_us":25262,"raft.queue_time_us":130,"raft.run_cpu_time_us":1238,"raft.run_wall_time_us":1237,"replication_time_us":17665}]]}
I20260809 14:30:15.960408 1020 tablet_replica.cc:330] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:15.961400 1020 raft_consensus.cc:2229] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:15.961952 1020 raft_consensus.cc:2258] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:15.964658 32133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31f9e3b82fa94588a07af146ed78f0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.964618 1009 raft_consensus.cc:482] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:15.965091 1009 raft_consensus.cc:504] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:15.965806 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd with delete_type TABLET_DATA_DELETED (replaced by tablet 23d2b217278c481e8464d00188c70048) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:15.967300 1009 leader_election.cc:288] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:15.967780 31943 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 47) took 2901 ms (client timeout 1000 ms). Trace:
W20260809 14:30:15.968134 31939 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet e74db9caa7f94059b1bba4a082fe57ef (table default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet e74db9caa7f94059b1bba4a082fe57ef consensus state has no leader
W20260809 14:30:15.968091 31943 rpcz_store.cc:269] 0809 14:30:13.066596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:13.579453 (+512857us) service_pool.cc:227] Handling call
0809 14:30:13.652999 (+ 73546us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:15.967720 (+2314721us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":144103,"reported_tablets":18,"spinlock_wait_cycles":2386816}
W20260809 14:30:15.969890 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 331626us
I20260809 14:30:15.970146 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f171260cb764ac284aef85e4219767e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:15.969843 31939 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 48) took 1738 ms (client timeout 1000 ms). Trace:
I20260809 14:30:15.971668 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d592b1c4520467791ffd9d36e0d97fd" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:15.972235 32513 raft_consensus.cc:3037] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:15.973578 32443 leader_election.cc:341] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:15.974012 32443 leader_election.cc:302] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:15.918427 31937 rpcz_store.cc:269] 0809 14:30:12.362426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.363855 (+ 1429us) service_pool.cc:227] Handling call
0809 14:30:13.148792 (+784937us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:15.918029 (+2769237us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:14.102117 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:14.102713 (+ 596us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:14.102747 (+ 34us) write_op.cc:435] Acquired schema lock
0809 14:30:14.102792 (+ 45us) tablet.cc:660] Decoding operations
0809 14:30:14.103550 (+ 758us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:14.103625 (+ 75us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:14.103661 (+ 36us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:14.103992 (+ 331us) tablet.cc:699] Row locks acquired
0809 14:30:14.104014 (+ 22us) write_op.cc:260] PREPARE: finished
0809 14:30:14.104203 (+ 189us) write_op.cc:270] Start()
0809 14:30:14.104390 (+ 187us) write_op.cc:276] Timestamp: P: 1786285814104150 usec, L: 0
0809 14:30:14.104420 (+ 30us) op_driver.cc:331] REPLICATION: starting
0809 14:30:14.105237 (+ 817us) log.cc:842] Serialized 533 byte log entry
0809 14:30:14.250464 (+145227us) op_driver.cc:444] REPLICATION: finished
0809 14:30:14.271641 (+ 21177us) write_op.cc:301] APPLY: starting
0809 14:30:14.271731 (+ 90us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:14.272773 (+ 1042us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:14.282803 (+ 10030us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:14.283359 (+ 556us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:14.283589 (+ 230us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:14.283693 (+ 104us) write_op.cc:312] APPLY: finished
0809 14:30:14.284021 (+ 328us) log.cc:842] Serialized 23 byte log entry
0809 14:30:14.284285 (+ 264us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.284376 (+ 91us) write_op.cc:454] Released schema lock
0809 14:30:14.284502 (+ 126us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:14.286579 (+ 2077us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:14.286622 (+ 43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39880,"reported_tablets":12,"spinlock_wait_cycles":1797208448,"child_traces":[["op",{"apply.queue_time_us":717,"apply.run_cpu_time_us":2878,"apply.run_wall_time_us":15262,"num_ops":1,"prepare.queue_time_us":240,"prepare.run_cpu_time_us":3171,"prepare.run_wall_time_us":11211,"replication_time_us":145949}]]}
I20260809 14:30:15.975119 939 raft_consensus.cc:2726] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:15.976197 31937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f72f5e4a712b4a519089aa13f2060a2a" } modify_external_catalogs: true
I20260809 14:30:15.977999 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c22055a5d334a57a6b8f2f8f9248d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.979244 32513 raft_consensus.cc:2454] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:15.980857 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af4e169f518447c8d29c630dca93ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.981313 31946 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_a6bdfc241dfa4c51a4dd388f95341565" } new_table_name: "default.Table_f13ec0ce721048f4aabef02eb1f5cc44" modify_external_catalogs: true
I20260809 14:30:15.982430 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_18bbb19abeca4f54a421b38edeba896c" } modify_external_catalogs: true
I20260809 14:30:15.984850 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b4c2a9dce646dda38647ef011b8260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:15.986019 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:15.987480 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 9922cad6d10945d5b013a168a3496676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:15.987728 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 1eb9d7c27381473aa87b718cca57b678) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:15.988301 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f171260cb764ac284aef85e4219767e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:15.988828 32310 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 with delete_type TABLET_DATA_DELETED (replaced by tablet b393017627934fa1b740d8ba8ac62d51) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:15.989152 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_61526cd7c2b14f0a916e4d916c607a78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:15.990545 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_61526cd7c2b14f0a916e4d916c607a78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:15.994077 31938 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 58) took 1511 ms (client timeout 1000 ms). Trace:
I20260809 14:30:16.014348 32691 tablet_bootstrap.cc:492] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:16.014884 32691 ts_tablet_manager.cc:1397] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.135s user 0.007s sys 0.008s
I20260809 14:30:16.009334 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d592b1c4520467791ffd9d36e0d97fd" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:15.970697 31939 rpcz_store.cc:269] 0809 14:30:14.231893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:14.288628 (+ 56735us) service_pool.cc:227] Handling call
0809 14:30:14.313543 (+ 24915us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:15.969769 (+1656226us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:15.692780 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:15.693189 (+ 409us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:15.693214 (+ 25us) write_op.cc:435] Acquired schema lock
0809 14:30:15.693231 (+ 17us) tablet.cc:660] Decoding operations
0809 14:30:15.693533 (+ 302us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:15.693573 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:15.693596 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:15.693758 (+ 162us) tablet.cc:699] Row locks acquired
0809 14:30:15.693774 (+ 16us) write_op.cc:260] PREPARE: finished
0809 14:30:15.693892 (+ 118us) write_op.cc:270] Start()
0809 14:30:15.693976 (+ 84us) write_op.cc:276] Timestamp: P: 1786285815693835 usec, L: 0
0809 14:30:15.693995 (+ 19us) op_driver.cc:331] REPLICATION: starting
0809 14:30:15.694504 (+ 509us) log.cc:842] Serialized 533 byte log entry
0809 14:30:15.718638 (+ 24134us) op_driver.cc:444] REPLICATION: finished
0809 14:30:15.727008 (+ 8370us) write_op.cc:301] APPLY: starting
0809 14:30:15.727090 (+ 82us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:15.727302 (+ 212us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:15.727445 (+ 143us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:15.727833 (+ 388us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:15.728160 (+ 327us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:15.729960 (+ 1800us) write_op.cc:312] APPLY: finished
0809 14:30:15.730335 (+ 375us) log.cc:842] Serialized 23 byte log entry
0809 14:30:15.731722 (+ 1387us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:15.731832 (+ 110us) write_op.cc:454] Released schema lock
0809 14:30:15.731892 (+ 60us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:15.732192 (+ 300us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:15.732244 (+ 52us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":21,"spinlock_wait_cycles":42293504,"child_traces":[["op",{"apply.queue_time_us":8145,"apply.run_cpu_time_us":2695,"apply.run_wall_time_us":5502,"num_ops":1,"prepare.queue_time_us":27845,"prepare.run_cpu_time_us":2155,"prepare.run_wall_time_us":42371,"raft.queue_time_us":1352,"raft.run_cpu_time_us":684,"raft.run_wall_time_us":742,"replication_time_us":24577}]]}
I20260809 14:30:16.019234 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c955b244a00c442fb7737b4bacb71ed6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:15.999547 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_18bbb19abeca4f54a421b38edeba896c" } modify_external_catalogs: true
I20260809 14:30:16.020725 32705 tablet_bootstrap.cc:492] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:16.020838 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c955b244a00c442fb7737b4bacb71ed6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:16.024379 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet bce7aec3225a4fd89b3bfd963b64582a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bce7aec3225a4fd89b3bfd963b64582a already in progress: creating tablet
W20260809 14:30:16.006783 31938 rpcz_store.cc:269] 0809 14:30:14.482850 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:14.721735 (+238885us) service_pool.cc:227] Handling call
0809 14:30:15.816662 (+1094927us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:15.816687 (+ 25us) write_op.cc:454] Released schema lock
0809 14:30:15.994015 (+177328us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:15.795793 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:15.796254 (+ 461us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:15.796284 (+ 30us) write_op.cc:435] Acquired schema lock
0809 14:30:15.796297 (+ 13us) tablet.cc:660] Decoding operations
0809 14:30:15.796706 (+ 409us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:15.796748 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:15.796775 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:15.797038 (+ 263us) tablet.cc:699] Row locks acquired
0809 14:30:15.797050 (+ 12us) write_op.cc:260] PREPARE: finished
0809 14:30:15.797163 (+ 113us) write_op.cc:270] Start()
0809 14:30:15.797248 (+ 85us) write_op.cc:276] Timestamp: P: 1786285815797104 usec, L: 0
0809 14:30:15.797263 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 14:30:15.797777 (+ 514us) log.cc:842] Serialized 913 byte log entry
0809 14:30:15.808868 (+ 11091us) op_driver.cc:444] REPLICATION: finished
0809 14:30:15.809196 (+ 328us) write_op.cc:301] APPLY: starting
0809 14:30:15.809248 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:15.809359 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:15.809472 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:15.809702 (+ 230us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:15.809835 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:15.809877 (+ 42us) write_op.cc:312] APPLY: finished
0809 14:30:15.810065 (+ 188us) log.cc:842] Serialized 29 byte log entry
0809 14:30:15.810211 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:15.810283 (+ 72us) write_op.cc:454] Released schema lock
0809 14:30:15.810314 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":37388,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":173,"apply.run_cpu_time_us":1326,"apply.run_wall_time_us":1325,"num_ops":2,"prepare.queue_time_us":4591,"prepare.run_cpu_time_us":2276,"prepare.run_wall_time_us":2276,"raft.queue_time_us":1272,"raft.run_cpu_time_us":668,"raft.run_wall_time_us":669,"replication_time_us":11527}]]}
I20260809 14:30:16.025589 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b62ea93ef63a4b0ba177bdae01cc95b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:16.026916 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b62ea93ef63a4b0ba177bdae01cc95b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.031003 32132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:16.031278 1020 ts_tablet_manager.cc:1905] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:16.045379 1004 tablet_replica.cc:330] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:16.046398 1004 raft_consensus.cc:2229] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:16.046985 1004 raft_consensus.cc:2258] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:16.049383 32168 raft_consensus.cc:3037] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:16.004705 31937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f72f5e4a712b4a519089aa13f2060a2a" } modify_external_catalogs: true
I20260809 14:30:15.999126 31940 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:49782 (request call id 298) took 4243 ms. Trace:
I20260809 14:30:16.054260 31940 rpcz_store.cc:276] 0809 14:30:11.755869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:12.063154 (+307285us) service_pool.cc:227] Handling call
0809 14:30:13.159407 (+1096253us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d97d6 00007ffa4b01f634 00007ffa4b01f5c2 00007ffa4b01f4f1 00007ffa4b01f46c 00007ffa4b01e722 00007ffa492b894c 00007ffa492b7d86
0809 14:30:13.180572 (+ 21165us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:13.183300 (+ 2728us) catalog_manager.cc:3616] Apply alter schema
0809 14:30:13.183870 (+ 570us) catalog_manager.cc:3639] Acquired catalog manager lock
0809 14:30:13.184069 (+ 199us) catalog_manager.cc:3786] Updating metadata on disk
0809 14:30:13.655824 (+471755us) catalog_manager.cc:3817] Committing alterations to in-memory state
0809 14:30:15.998981 (+2343157us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:13.390124 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:13.391221 (+ 1097us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:13.391288 (+ 67us) write_op.cc:435] Acquired schema lock
0809 14:30:13.391337 (+ 49us) tablet.cc:660] Decoding operations
0809 14:30:13.392068 (+ 731us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:13.392158 (+ 90us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:13.392252 (+ 94us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:13.440719 (+ 48467us) tablet.cc:699] Row locks acquired
0809 14:30:13.440772 (+ 53us) write_op.cc:260] PREPARE: finished
0809 14:30:13.441114 (+ 342us) write_op.cc:270] Start()
0809 14:30:13.441278 (+ 164us) write_op.cc:276] Timestamp: P: 1786285813441008 usec, L: 0
0809 14:30:13.441326 (+ 48us) op_driver.cc:331] REPLICATION: starting
0809 14:30:13.452508 (+ 11182us) log.cc:842] Serialized 405 byte log entry
0809 14:30:13.455694 (+ 3186us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48e471 00007ffa4b505198 00007ffa4b966ade 00007ffa4b9662b8 00007ffa4b96aba1 00007ffa4b96ab59 00007ffa4b96aae9 00007ffa4b96aab1 00007ffa4b969dad 0000000000580784 00000000005805b9
0809 14:30:13.652752 (+197058us) op_driver.cc:444] REPLICATION: finished
0809 14:30:13.654490 (+ 1738us) write_op.cc:301] APPLY: starting
0809 14:30:13.654548 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:13.654632 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:13.654655 (+ 23us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:13.654877 (+ 222us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:13.655000 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:13.655151 (+ 151us) write_op.cc:312] APPLY: finished
0809 14:30:13.655353 (+ 202us) log.cc:842] Serialized 23 byte log entry
0809 14:30:13.655519 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.655590 (+ 71us) write_op.cc:454] Released schema lock
0809 14:30:13.655625 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:13.655823 (+ 198us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:13.655851 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":2519175552,"child_traces":[["op",{"apply.queue_time_us":1552,"apply.run_cpu_time_us":1594,"apply.run_wall_time_us":1593,"num_ops":1,"prepare.queue_time_us":159428,"prepare.run_cpu_time_us":6659,"prepare.run_wall_time_us":95246,"replication_time_us":211308,"spinlock_wait_cycles":23607680}]]}
W20260809 14:30:16.058821 31946 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5ac7afd531a746cab7618ead98b4c4d9 consensus state has no leader
I20260809 14:30:16.059259 31946 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:49782 (request call id 323) took 1136 ms. Trace:
I20260809 14:30:16.060084 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_0e45100c9aee47e5a5a225d0f0564718"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:16.061609 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e45100c9aee47e5a5a225d0f0564718 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.065752 32163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74db9caa7f94059b1bba4a082fe57ef" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:16.070261 1004 ts_tablet_manager.cc:1905] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:16.085078 32705 tablet_bootstrap.cc:654] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.085279 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet e846e795144e48e3b2a1eca8c1b26489
I20260809 14:30:16.090101 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet e846e795144e48e3b2a1eca8c1b26489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:16.091001 32296 tablet_service.cc:1514] Processing DeleteTablet for tablet e846e795144e48e3b2a1eca8c1b26489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:16.094250 32260 leader_election.cc:341] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.107187 32693 tablet_bootstrap.cc:492] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:16.107702 32693 ts_tablet_manager.cc:1397] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.744s user 0.019s sys 0.000s
W20260809 14:30:16.108791 32260 leader_election.cc:341] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.110018 32693 raft_consensus.cc:348] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.110772 32693 raft_consensus.cc:374] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.111078 32693 raft_consensus.cc:729] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:30:16.111246 32260 consensus_peers.cc:487] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:16.116495 32141 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 236) took 1120 ms. Trace:
I20260809 14:30:16.116815 32141 rpcz_store.cc:276] 0809 14:30:14.996394 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.230797 (+234403us) service_pool.cc:227] Handling call
0809 14:30:15.232293 (+ 1496us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:15.232363 (+ 70us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:15.805833 (+573470us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:15.839700 (+ 33867us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:16.116021 (+276321us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800001898. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c00ebf11 00007fb6c026f851 00007fb6c026fe82 00007fb6c912f8b9 00007fb6c81635d8 00007fb6c8227063 00007fb6c8222e2f 00007fb6c82235ed 00007fb6c81d4869 00007fb6c68c6c44 00007fb6c68c6bd2
0809 14:30:16.116285 (+ 264us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":24161024}
I20260809 14:30:16.059540 31946 rpcz_store.cc:276] 0809 14:30:14.923167 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.954920 (+1031753us) service_pool.cc:227] Handling call
0809 14:30:15.981229 (+ 26309us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d97d6 00007ffa4b01f634 00007ffa4b01f5c2 00007ffa4b01f4f1 00007ffa4b01f46c 00007ffa4b01e722 00007ffa492b894c 00007ffa492b7d86
0809 14:30:16.005363 (+ 24134us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:16.006205 (+ 842us) catalog_manager.cc:3616] Apply alter schema
0809 14:30:16.006848 (+ 643us) catalog_manager.cc:3639] Acquired catalog manager lock
0809 14:30:16.009363 (+ 2515us) catalog_manager.cc:3786] Updating metadata on disk
0809 14:30:16.058164 (+ 48801us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:16.058188 (+ 24us) write_op.cc:454] Released schema lock
0809 14:30:16.058384 (+ 196us) catalog_manager.cc:3817] Committing alterations to in-memory state
0809 14:30:16.059213 (+ 829us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:16.012276 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:16.012770 (+ 494us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:16.012807 (+ 37us) write_op.cc:435] Acquired schema lock
0809 14:30:16.012826 (+ 19us) tablet.cc:660] Decoding operations
0809 14:30:16.013129 (+ 303us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:16.013178 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:16.013202 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:16.013377 (+ 175us) tablet.cc:699] Row locks acquired
0809 14:30:16.013394 (+ 17us) write_op.cc:260] PREPARE: finished
0809 14:30:16.013525 (+ 131us) write_op.cc:270] Start()
0809 14:30:16.013625 (+ 100us) write_op.cc:276] Timestamp: P: 1786285816013446 usec, L: 0
0809 14:30:16.013646 (+ 21us) op_driver.cc:331] REPLICATION: starting
0809 14:30:16.014155 (+ 509us) log.cc:842] Serialized 405 byte log entry
0809 14:30:16.023970 (+ 9815us) op_driver.cc:444] REPLICATION: finished
0809 14:30:16.024302 (+ 332us) write_op.cc:301] APPLY: starting
0809 14:30:16.024351 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:16.024439 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:16.024584 (+ 145us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:16.024720 (+ 136us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:16.024843 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:16.024901 (+ 58us) write_op.cc:312] APPLY: finished
0809 14:30:16.025090 (+ 189us) log.cc:842] Serialized 23 byte log entry
0809 14:30:16.025228 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:16.025286 (+ 58us) write_op.cc:454] Released schema lock
0809 14:30:16.025313 (+ 27us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":58678528,"child_traces":[["op",{"apply.queue_time_us":194,"apply.run_cpu_time_us":1177,"apply.run_wall_time_us":1178,"num_ops":1,"prepare.queue_time_us":1557,"prepare.run_cpu_time_us":2165,"prepare.run_wall_time_us":2172,"raft.queue_time_us":1197,"raft.run_cpu_time_us":714,"raft.run_wall_time_us":713,"replication_time_us":10244}]]}
W20260809 14:30:16.120114 32099 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.730s user 0.006s sys 0.007s
W20260809 14:30:16.120507 32099 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.730s user 0.007s sys 0.007s
W20260809 14:30:16.122496 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 845362us
I20260809 14:30:16.057852 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74db9caa7f94059b1bba4a082fe57ef" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:16.123955 32515 raft_consensus.cc:2454] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:16.088948 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet e846e795144e48e3b2a1eca8c1b26489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:16.124620 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:16.125612 32326 raft_consensus.cc:2379] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:16.142846 32691 raft_consensus.cc:348] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.111894 32693 consensus_queue.cc:260] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:16.111770 32260 consensus_peers.cc:487] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:16.152971 32260 consensus_peers.cc:487] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:16.153709 32260 leader_election.cc:341] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.149258 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 9922cad6d10945d5b013a168a3496676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:16.157791 32691 raft_consensus.cc:374] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.158134 32691 raft_consensus.cc:729] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:30:16.161881 1001 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 1495769309838729370) tablet 00000000000000000000000000000000 cur holder 0x7b5800060900
I20260809 14:30:16.164193 32693 ts_tablet_manager.cc:1428] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.056s user 0.004s sys 0.000s
I20260809 14:30:16.174460 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet bce7aec3225a4fd89b3bfd963b64582a with delete_type TABLET_DATA_DELETED (replaced by tablet 408ee42ea5844741b9ab514cbe6a94e4) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:16.175792 935 raft_consensus.cc:482] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:16.176298 935 raft_consensus.cc:504] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.176842 32691 consensus_queue.cc:260] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.179234 32691 ts_tablet_manager.cc:1428] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.055s user 0.003s sys 0.002s
I20260809 14:30:16.180135 935 leader_election.cc:288] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:16.181329 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:16.181888 32508 raft_consensus.cc:2454] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f94e0d792c46c9a0b6ab42820da6eb in term 1.
I20260809 14:30:16.183652 32095 leader_election.cc:302] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:16.184476 935 raft_consensus.cc:2781] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:30:16.185294 32691 tablet_bootstrap.cc:492] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:16.185182 935 raft_consensus.cc:482] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:16.185644 935 raft_consensus.cc:3037] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:16.199712 32691 tablet_bootstrap.cc:654] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.208369 996 ts_tablet_manager.cc:1918] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:16.210721 996 log.cc:1196] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/94c4cbdda4ed48a8b34728d231497d02
I20260809 14:30:16.212342 32693 tablet_bootstrap.cc:492] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:16.213322 996 ts_tablet_manager.cc:1939] T 94c4cbdda4ed48a8b34728d231497d02 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:16.220897 996 tablet_replica.cc:330] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:16.221083 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 94c4cbdda4ed48a8b34728d231497d02 (table default.Table_bb45fcc479b0477f872971659421b3ac [id=7043672a284145c7a5c9a5d80c322ef8]) successfully deleted
I20260809 14:30:16.221830 996 raft_consensus.cc:2229] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:16.222323 996 raft_consensus.cc:2258] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:16.224161 996 ts_tablet_manager.cc:1905] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:16.218165 32691 tablet_bootstrap.cc:492] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:16.224700 32691 ts_tablet_manager.cc:1397] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.006s
I20260809 14:30:16.227015 32691 raft_consensus.cc:348] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.227658 32691 raft_consensus.cc:374] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.231794 32691 raft_consensus.cc:729] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:16.232532 32691 consensus_queue.cc:260] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.234663 32691 ts_tablet_manager.cc:1428] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260809 14:30:16.235497 32691 tablet_bootstrap.cc:492] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:16.236523 32693 tablet_bootstrap.cc:654] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.241334 32260 leader_election.cc:302] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:16.252563 1016 raft_consensus.cc:2781] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:16.253085 1016 raft_consensus.cc:686] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:16.253391 1011 consensus_queue.cc:1035] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:16.253909 1016 consensus_queue.cc:237] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.257799 32260 leader_election.cc:302] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:16.257894 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_ac82ee262e4a40bcb9bc60cc74a8861f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:16.258641 1011 raft_consensus.cc:2781] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:16.258991 1011 raft_consensus.cc:482] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:16.259335 1011 raft_consensus.cc:3037] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:16.259608 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac82ee262e4a40bcb9bc60cc74a8861f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.262174 31946 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f924e07646a94fd7a82006cef63a8abe" } new_table_name: "default.Table_24df220422ea4f1ab4d5189b0b5ffc80" modify_external_catalogs: true
I20260809 14:30:16.277333 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:16.286358 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b219214ad9f94dd991137e6a5a35764c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:16.287779 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b219214ad9f94dd991137e6a5a35764c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.289670 31937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_d4baddb55ef84a0f9456db8bb8782728" } modify_external_catalogs: true
I20260809 14:30:16.290166 31937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_d4baddb55ef84a0f9456db8bb8782728" } modify_external_catalogs: true
I20260809 14:30:16.293010 939 raft_consensus.cc:482] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:16.293419 939 raft_consensus.cc:504] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.295275 939 leader_election.cc:288] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:16.296805 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdfc56edcd34696a7b806b0ee34d594" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:16.298157 32443 leader_election.cc:341] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.300707 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e6685adbd7a7408f9fc60d67696e3b04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:16.302014 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6685adbd7a7408f9fc60d67696e3b04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.268344 32691 tablet_bootstrap.cc:654] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.309342 1011 raft_consensus.cc:504] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.311336 1011 leader_election.cc:288] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:16.312533 1022 consensus_queue.cc:1035] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:16.313079 1007 consensus_queue.cc:1035] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:16.315949 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdfc56edcd34696a7b806b0ee34d594" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:16.319060 32439 leader_election.cc:341] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.319522 32439 leader_election.cc:302] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:16.320188 31937 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f23f59619d15410b83e1f15dd85966d9
I20260809 14:30:16.320236 939 raft_consensus.cc:2726] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:16.325316 1019 raft_consensus.cc:482] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:16.325824 1019 raft_consensus.cc:504] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.328017 1019 leader_election.cc:288] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:16.328326 31937 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_478fce1bf3d54075835a5d464370d13e" } new_table_name: "default.Table_c99fa72bd1bd441aaa57d3f32a1d1d9e" modify_external_catalogs: true
I20260809 14:30:16.335750 32099 leader_election.cc:302] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:16.344833 996 ts_tablet_manager.cc:1918] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:16.345248 996 log.cc:1196] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/79bb4dd74afe4ace94eb1519950a83c0
I20260809 14:30:16.345916 996 ts_tablet_manager.cc:1939] T 79bb4dd74afe4ace94eb1519950a83c0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:16.347100 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 79bb4dd74afe4ace94eb1519950a83c0 (table default.Table_13a67d50c6c746e99d3c0ce47b81f6ad [id=20f2c60561964f50889f993f7a9421f1]) successfully deleted
I20260809 14:30:16.347224 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:16.358816 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74db9caa7f94059b1bba4a082fe57ef" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:16.359345 32513 raft_consensus.cc:3037] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:16.362103 1014 raft_consensus.cc:3037] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:16.366416 32513 raft_consensus.cc:2454] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:16.370666 1040 tablet.cc:1722] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:16.371093 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74db9caa7f94059b1bba4a082fe57ef" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:16.373359 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 29fdaef8a95e4aa0b23018d384442d16 because the tablet was not found. No further retry: Not found: Tablet not found: 29fdaef8a95e4aa0b23018d384442d16
I20260809 14:30:16.375705 996 tablet_replica.cc:330] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:16.377859 996 raft_consensus.cc:2229] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:16.381935 996 raft_consensus.cc:2258] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:30:16.392515 31937 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 46d8d00bbf2e4f4289448e48536979ea consensus state has no leader
I20260809 14:30:16.388238 996 ts_tablet_manager.cc:1905] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:16.407378 939 raft_consensus.cc:482] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:16.407931 939 raft_consensus.cc:504] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.408973 32691 tablet_bootstrap.cc:492] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:16.409349 32691 ts_tablet_manager.cc:1397] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.174s user 0.017s sys 0.008s
I20260809 14:30:16.412921 939 leader_election.cc:288] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:16.413522 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ff6ff27f6144d08d68a5e79c12963f" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:16.413630 32691 raft_consensus.cc:348] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.414247 32691 raft_consensus.cc:374] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.414498 32691 raft_consensus.cc:729] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:30:16.414880 32443 leader_election.cc:341] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.415028 32691 consensus_queue.cc:260] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.418406 32693 tablet_bootstrap.cc:492] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:16.418933 32693 ts_tablet_manager.cc:1397] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.207s user 0.010s sys 0.000s
I20260809 14:30:16.373292 1039 tablet.cc:1722] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260809 14:30:16.425021 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 89369.6us
I20260809 14:30:16.426195 32691 ts_tablet_manager.cc:1428] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.017s user 0.001s sys 0.004s
I20260809 14:30:16.427091 32691 tablet_bootstrap.cc:492] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:16.430935 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b341959508524bd796f68f68e26f6c82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:16.432536 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b341959508524bd796f68f68e26f6c82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.423708 1020 ts_tablet_manager.cc:1918] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:16.444031 1020 log.cc:1196] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/bce7aec3225a4fd89b3bfd963b64582a
I20260809 14:30:16.445062 1020 ts_tablet_manager.cc:1939] T bce7aec3225a4fd89b3bfd963b64582a P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:16.363945 935 raft_consensus.cc:504] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.447466 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88644048fa4a443fac09e951d78b3063" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:16.454325 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ff6ff27f6144d08d68a5e79c12963f" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:16.456245 32439 leader_election.cc:341] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.456686 32439 leader_election.cc:302] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:16.457639 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88644048fa4a443fac09e951d78b3063" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:16.463891 32705 tablet_bootstrap.cc:492] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:16.464326 32705 ts_tablet_manager.cc:1397] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.446s user 0.014s sys 0.011s
I20260809 14:30:16.460568 939 raft_consensus.cc:2726] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:16.468866 32135 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 238) took 1218 ms. Trace:
I20260809 14:30:16.471947 32691 tablet_bootstrap.cc:654] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.484081 1014 raft_consensus.cc:2726] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:16.485319 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet bce7aec3225a4fd89b3bfd963b64582a (table default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e]) successfully deleted
I20260809 14:30:16.485539 935 leader_election.cc:288] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 election: Requested vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:16.486426 1020 tablet_replica.cc:330] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:16.469198 32135 rpcz_store.cc:276] 0809 14:30:15.231731 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.231919 (+ 188us) service_pool.cc:227] Handling call
0809 14:30:15.234557 (+ 2638us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:15.234743 (+ 186us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:15.940171 (+705428us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:16.392338 (+452167us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:16.446098 (+ 53760us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:16.492381 32137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 242) took 1259 ms. Trace:
W20260809 14:30:16.495280 32095 leader_election.cc:341] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 79bb4dd74afe4ace94eb1519950a83c0
I20260809 14:30:16.492726 32137 rpcz_store.cc:276] 0809 14:30:15.233155 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.275039 (+ 41884us) service_pool.cc:227] Handling call
0809 14:30:15.968507 (+693468us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:15.968634 (+ 127us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:16.182333 (+213699us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:16.411407 (+229074us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:16.492320 (+ 80913us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:16.498502 32513 raft_consensus.cc:2454] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:16.507863 1004 ts_tablet_manager.cc:1918] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:16.511884 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb4dd74afe4ace94eb1519950a83c0" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:16.498574 1049 tablet.cc:1722] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:16.507452 996 ts_tablet_manager.cc:1918] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:16.516703 996 log.cc:1196] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/3d592b1c4520467791ffd9d36e0d97fd
I20260809 14:30:16.517383 996 ts_tablet_manager.cc:1939] T 3d592b1c4520467791ffd9d36e0d97fd P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:16.518375 32693 raft_consensus.cc:348] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.519306 32693 raft_consensus.cc:374] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.519528 32693 raft_consensus.cc:729] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:16.519805 996 tablet_replica.cc:330] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:16.520902 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 3d592b1c4520467791ffd9d36e0d97fd (table default.Table_45d346a083d04fc6b8a9d8631c65bff1 [id=4a72c13a29f74ba19c69e2392c122b72]) successfully deleted
I20260809 14:30:16.521596 996 raft_consensus.cc:2229] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 14:30:16.477566 32260 leader_election.cc:341] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.478003 1016 consensus_queue.cc:1035] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:16.522020 996 pending_rounds.cc:62] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Trying to abort 1 pending ops.
I20260809 14:30:16.522250 996 pending_rounds.cc:68] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626697333719040 op_type: NO_OP noop_request { }
I20260809 14:30:16.522645 996 raft_consensus.cc:2866] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:16.523006 996 raft_consensus.cc:2258] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:30:16.523110 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 45577.3us
I20260809 14:30:16.523622 31937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e58ccd00bec645e4be0b92281bbcc459" } modify_external_catalogs: true
I20260809 14:30:16.524194 31937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e58ccd00bec645e4be0b92281bbcc459" } modify_external_catalogs: true
I20260809 14:30:16.526204 996 ts_tablet_manager.cc:1905] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:16.526641 1049 tablet.cc:1722] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 14:30:16.508279 1004 log.cc:1196] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/b0ad3946f5de4387af3627a88b6404cf
I20260809 14:30:16.532163 1004 ts_tablet_manager.cc:1939] T b0ad3946f5de4387af3627a88b6404cf P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:16.541733 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_d7a3df21ac0a4fd2b7cb295edfb62f96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:16.543138 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet b0ad3946f5de4387af3627a88b6404cf (table default.Table_bb02de675fd247e5af1ff74a6c57f2bd [id=eb07bcad144c4719ae97a233970493f8]) successfully deleted
W20260809 14:30:16.543270 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7a3df21ac0a4fd2b7cb295edfb62f96 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:16.523823 32260 leader_election.cc:341] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.548861 32099 leader_election.cc:302] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:16.550590 1004 tablet_replica.cc:330] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:16.551051 1020 raft_consensus.cc:2229] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:16.551496 1004 raft_consensus.cc:2229] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:16.551671 1020 pending_rounds.cc:62] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Trying to abort 1 pending ops.
I20260809 14:30:16.551877 1020 pending_rounds.cc:68] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626697312960512 op_type: NO_OP noop_request { }
I20260809 14:30:16.552237 1020 raft_consensus.cc:2866] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:16.552565 1020 raft_consensus.cc:2258] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:16.552762 1014 raft_consensus.cc:2726] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 14:30:16.554409 1020 ts_tablet_manager.cc:1905] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:16.561072 32693 consensus_queue.cc:260] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.574952 32691 tablet_bootstrap.cc:492] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:16.575547 32691 ts_tablet_manager.cc:1397] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.149s user 0.003s sys 0.010s
W20260809 14:30:16.579737 32260 consensus_peers.cc:487] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Couldn't send request to peer 6bed14aff53a41b2bbf5a13ce55dec5a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94c4cbdda4ed48a8b34728d231497d02. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:16.466702 32705 raft_consensus.cc:348] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.591987 32165 raft_consensus.cc:1226] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Rejecting Update request from peer 3eb0bc287b12421d89abf97ca66e4d9b for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260809 14:30:16.598428 32691 raft_consensus.cc:348] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.601200 32691 raft_consensus.cc:374] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.601460 32691 raft_consensus.cc:729] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:16.602077 32691 consensus_queue.cc:260] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.603063 32260 leader_election.cc:302] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:16.604020 1016 raft_consensus.cc:2781] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:16.586952 32705 raft_consensus.cc:374] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.604583 1016 raft_consensus.cc:686] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:16.604702 32705 raft_consensus.cc:729] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:16.605713 1016 consensus_queue.cc:237] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.614533 32693 ts_tablet_manager.cc:1428] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.195s user 0.007s sys 0.000s
I20260809 14:30:16.615418 32693 tablet_bootstrap.cc:492] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:16.630975 32691 ts_tablet_manager.cc:1428] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.055s user 0.008s sys 0.000s
I20260809 14:30:16.631763 32691 tablet_bootstrap.cc:492] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:16.632613 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:16.633910 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_01106b7a16e142518d374e93aac4da95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:16.635495 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01106b7a16e142518d374e93aac4da95 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.638206 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9e998648ceee4a19835ede64b0ef1038" } new_table_name: "default.Table_6d1220a2972747a7b2050e1714d86647" modify_external_catalogs: true
I20260809 14:30:16.644147 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5a11d072d3c94cccb6823afc0b64eb83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:16.626173 32705 consensus_queue.cc:260] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:16.655417 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a11d072d3c94cccb6823afc0b64eb83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.658536 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_d50800cad30d400ba6b4bb95b5f60042"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:16.660140 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d50800cad30d400ba6b4bb95b5f60042 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:16.673045 1050 log.cc:924] Time spent T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.085s user 0.000s sys 0.002s
I20260809 14:30:16.673401 1039 tablet.cc:1722] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:16.646876 1004 raft_consensus.cc:2258] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:16.693802 32260 leader_election.cc:302] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:16.699112 1004 ts_tablet_manager.cc:1905] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:16.702819 31938 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60acf8a11d6143948bfe0040f74f5ef8 consensus state has no leader
I20260809 14:30:16.707607 32691 tablet_bootstrap.cc:654] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.707502 1011 raft_consensus.cc:2781] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:16.708168 1011 raft_consensus.cc:482] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:16.708803 1011 raft_consensus.cc:3037] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:16.715975 32705 ts_tablet_manager.cc:1428] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.251s user 0.005s sys 0.001s
I20260809 14:30:16.721143 1040 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47010 (request call id 238) took 1060 ms. Trace:
I20260809 14:30:16.725183 1016 raft_consensus.cc:482] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:16.725827 1016 raft_consensus.cc:504] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.754297 32705 tablet_bootstrap.cc:492] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:16.764202 1016 leader_election.cc:288] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:16.765703 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 71978.3us
I20260809 14:30:16.768211 1060 tablet.cc:1722] T 5121e9f5a85a45d285d78f41a442769a P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:16.771302 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_4eccea50c485475c9e4a6f9e9990cac1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:16.772431 1049 tablet.cc:1722] T 5121e9f5a85a45d285d78f41a442769a P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260809 14:30:16.772948 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4eccea50c485475c9e4a6f9e9990cac1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:16.781224 32691 tablet_bootstrap.cc:492] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:16.781648 32691 ts_tablet_manager.cc:1397] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.150s user 0.013s sys 0.001s
I20260809 14:30:16.790166 1019 consensus_queue.cc:1035] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:16.792510 32691 raft_consensus.cc:348] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.793175 32691 raft_consensus.cc:374] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.793555 32691 raft_consensus.cc:729] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:16.794134 32691 consensus_queue.cc:260] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.721736 1040 rpcz_store.cc:276] 0809 14:30:15.662577 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.663768 (+ 1191us) service_pool.cc:227] Handling call
0809 14:30:16.721012 (+1057244us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:15.875592 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:15.876003 (+ 411us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:15.876040 (+ 37us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:15.876120 (+ 80us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:15.876252 (+ 132us) alter_schema_op.cc:126] START. Timestamp: 1786285815875362
0809 14:30:15.876280 (+ 28us) op_driver.cc:331] REPLICATION: starting
0809 14:30:15.876909 (+ 629us) log.cc:842] Serialized 173 byte log entry
0809 14:30:16.314603 (+437694us) op_driver.cc:444] REPLICATION: finished
0809 14:30:16.370434 (+ 55831us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:16.372173 (+ 1739us) log.cc:842] Serialized 5 byte log entry
0809 14:30:16.703072 (+330899us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:16.719258 (+ 16186us) log.cc:842] Serialized 14 byte log entry
0809 14:30:16.720773 (+ 1515us) alter_schema_op.cc:71] Released schema lock
0809 14:30:16.720800 (+ 27us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":55658,"prepare.queue_time_us":210902,"prepare.run_cpu_time_us":10462,"prepare.run_wall_time_us":108896,"replication_time_us":438243,"thread_start_us":441041,"threads_started":3,"wal-append.queue_time_us":80384}]]}
I20260809 14:30:16.799623 1040 tablet.cc:1722] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:16.802801 1020 ts_tablet_manager.cc:1918] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:16.803284 1020 log.cc:1196] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/79bb4dd74afe4ace94eb1519950a83c0
I20260809 14:30:16.804224 1020 ts_tablet_manager.cc:1939] T 79bb4dd74afe4ace94eb1519950a83c0 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:16.816984 32163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:16.817719 32163 raft_consensus.cc:2454] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:16.796290 32691 ts_tablet_manager.cc:1428] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260809 14:30:16.828727 32691 tablet_bootstrap.cc:492] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:16.819501 1009 raft_consensus.cc:482] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:16.832034 1009 raft_consensus.cc:504] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.833945 1009 leader_election.cc:288] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:16.834651 1020 tablet_replica.cc:330] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:16.835777 1020 raft_consensus.cc:2229] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:16.836138 996 ts_tablet_manager.cc:1918] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260809 14:30:16.836244 1058 log.cc:924] Time spent T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b: Append to log took a long time: real 0.208s user 0.001s sys 0.000s
I20260809 14:30:16.820892 32693 tablet_bootstrap.cc:654] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.847276 32260 leader_election.cc:302] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:16.847946 1016 raft_consensus.cc:2781] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:30:16.848268 1016 raft_consensus.cc:482] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:16.848692 1016 raft_consensus.cc:3037] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:16.861589 1020 raft_consensus.cc:2258] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:30:16.871762 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 60acf8a11d6143948bfe0040f74f5ef8 (table default.Table_6d1220a2972747a7b2050e1714d86647 [id=217a91e1a58b443d93738772edc80b76], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 60acf8a11d6143948bfe0040f74f5ef8 consensus state has no leader
I20260809 14:30:16.874270 1020 ts_tablet_manager.cc:1905] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:16.875325 31933 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.125:42169, user_credentials={real_user=slave}} blocked reactor thread for 74120us
I20260809 14:30:16.876288 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 79bb4dd74afe4ace94eb1519950a83c0 (table default.Table_13a67d50c6c746e99d3c0ce47b81f6ad [id=20f2c60561964f50889f993f7a9421f1]) successfully deleted
I20260809 14:30:16.887120 32693 tablet_bootstrap.cc:492] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:16.887753 32693 ts_tablet_manager.cc:1397] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.272s user 0.011s sys 0.000s
I20260809 14:30:16.890707 32693 raft_consensus.cc:348] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.891435 32693 raft_consensus.cc:374] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:16.891757 32693 raft_consensus.cc:729] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:16.892376 32693 consensus_queue.cc:260] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.895406 32693 ts_tablet_manager.cc:1428] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260809 14:30:16.911729 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5fdc5800b255423da5d44ad4ab4a1c2a" } new_table_name: "default.Table_20cad555a34f4f70a19814ffcb820bbe" modify_external_catalogs: true
I20260809 14:30:16.913389 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_8eb42cdceb174196b7743f3d3d12edae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:16.929076 32705 tablet_bootstrap.cc:654] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.945449 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cb7842617a4779bf110e4ea254e011" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:16.843371 32691 tablet_bootstrap.cc:654] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:16.924765 1049 tablet.cc:1722] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:16.954382 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_e6381d3c953a4958ada0a38267050cc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:16.958264 1060 tablet.cc:1722] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 14:30:16.973970 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cb7842617a4779bf110e4ea254e011" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:16.974731 32513 raft_consensus.cc:2454] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:16.946907 996 log.cc:1196] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/2f171260cb764ac284aef85e4219767e
I20260809 14:30:16.979960 996 ts_tablet_manager.cc:1939] T 2f171260cb764ac284aef85e4219767e P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:16.878176 1011 raft_consensus.cc:504] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:16.983855 32693 tablet_bootstrap.cc:492] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:16.981909 1016 raft_consensus.cc:504] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:16.996491 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6381d3c953a4958ada0a38267050cc0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.002111 1016 leader_election.cc:288] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:17.003845 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8eb42cdceb174196b7743f3d3d12edae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.012969 1011 leader_election.cc:288] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:17.004629 32705 tablet_bootstrap.cc:492] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:17.017412 32705 ts_tablet_manager.cc:1397] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.263s user 0.012s sys 0.001s
I20260809 14:30:17.018411 1048 raft_consensus.cc:482] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:17.019157 1048 raft_consensus.cc:504] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.021617 32705 raft_consensus.cc:348] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.022372 32705 raft_consensus.cc:374] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:17.022706 32705 raft_consensus.cc:729] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:17.024571 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "301b53eeb3be41a2b8ff2598d1b531d7" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:17.025972 1048 leader_election.cc:288] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:17.027220 32569 heartbeater.cc:347] Connected to a master server at 127.30.252.126:43223
I20260809 14:30:17.027772 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:17.033902 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "301b53eeb3be41a2b8ff2598d1b531d7" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:17.035337 32260 consensus_peers.cc:487] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Couldn't send request to peer 6bed14aff53a41b2bbf5a13ce55dec5a. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:17.041462 32260 leader_election.cc:341] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 29fdaef8a95e4aa0b23018d384442d16
W20260809 14:30:17.043115 32439 leader_election.cc:341] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:16.989583 1068 tablet.cc:1722] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:17.052129 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_7f6aaf330b564e2c8a3126bca9401698"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.053774 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f6aaf330b564e2c8a3126bca9401698 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.057199 32224 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:17.059118 32260 leader_election.cc:302] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:17.068230 32691 tablet_bootstrap.cc:492] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:17.069675 32691 ts_tablet_manager.cc:1397] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.241s user 0.017s sys 0.005s
I20260809 14:30:17.056720 1072 tablet.cc:1722] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260809 14:30:17.075387 32691 raft_consensus.cc:348] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.076041 32691 raft_consensus.cc:374] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:17.076303 32691 raft_consensus.cc:729] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:17.077046 32691 consensus_queue.cc:260] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.081619 32691 ts_tablet_manager.cc:1428] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260809 14:30:17.082615 32691 tablet_bootstrap.cc:492] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:17.088876 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 249) took 1119 ms. Trace:
I20260809 14:30:17.089674 996 rpcz_store.cc:276] 0809 14:30:15.969746 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.970013 (+ 267us) service_pool.cc:227] Handling call
0809 14:30:16.519654 (+549641us) ts_tablet_manager.cc:1145] Deleting tablet 2f171260cb764ac284aef85e4219767e
0809 14:30:16.519694 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:16.626161 (+106467us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:16.835882 (+209721us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:17.088782 (+252900us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":549109}
I20260809 14:30:17.091500 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 2f171260cb764ac284aef85e4219767e (table default.Table_42864987ccd14c05a6909a6e1527e208 [id=032f6264a2c842f1ba6b007331ccfcc6]) successfully deleted
W20260809 14:30:17.093468 32443 leader_election.cc:341] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:17.093822 32443 leader_election.cc:302] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:17.094983 1048 raft_consensus.cc:2726] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:17.107750 1009 raft_consensus.cc:2781] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:17.107983 32163 raft_consensus.cc:3037] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:17.108189 1009 raft_consensus.cc:482] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:17.108578 1009 raft_consensus.cc:3037] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:17.113298 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fdaef8a95e4aa0b23018d384442d16" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:17.114056 32165 raft_consensus.cc:3037] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:17.114550 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:17.115149 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_02301a82edfd45bd9511edf923e1028d [id=c1935d3bc49c405abe4d006932820224]). Expected version 3 got 0
I20260809 14:30:17.115679 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:17.117010 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88644048fa4a443fac09e951d78b3063" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:17.142681 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88644048fa4a443fac09e951d78b3063" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:17.143407 32513 raft_consensus.cc:3037] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:17.116173 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:17.125600 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_85987fca61094acbb4a924e9ceeec96a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.144899 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85987fca61094acbb4a924e9ceeec96a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.148909 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:17.157219 996 tablet_replica.cc:330] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:17.158571 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:17.159171 31948 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
W20260809 14:30:17.164558 1001 lock_manager.cc:586] Waited 2 seconds to obtain row lock on key '<redacted>' (key hash 1495769309838729370) tablet 00000000000000000000000000000000 cur holder 0x7b5800060900
I20260809 14:30:17.168107 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 197b996294454f22a98b37dab6bb2b1a (DEFAULT_TABLE table=default.Table_83c723c78d1b466f96b992e07866b7e0 [id=345fabe93b8b4d038d0faf2de4dc8556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.169248 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197b996294454f22a98b37dab6bb2b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.177048 32513 raft_consensus.cc:2454] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:17.179965 31939 master_service.cc:946] ReplaceTablet: received request to replace tablet ac27f377aa824654b6491c3065ef819c from {username='slave'} at 127.0.0.1:49782
W20260809 14:30:17.181761 32260 leader_election.cc:341] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:17.185420 32260 leader_election.cc:341] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:17.188184 32260 leader_election.cc:341] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 29fdaef8a95e4aa0b23018d384442d16
I20260809 14:30:17.188854 32260 leader_election.cc:302] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:17.195782 1009 raft_consensus.cc:504] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.198632 1009 leader_election.cc:288] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:17.199232 1009 raft_consensus.cc:2781] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:17.200541 32705 consensus_queue.cc:260] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.172309 996 raft_consensus.cc:2229] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:17.208014 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 5ac7afd531a746cab7618ead98b4c4d9 (table default.Table_f13ec0ce721048f4aabef02eb1f5cc44 [id=617231ca8be54c389a6c0d821b081eb2]). Expected version 1 got 0
I20260809 14:30:17.209100 996 raft_consensus.cc:2258] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:17.209558 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 94ff6ff27f6144d08d68a5e79c12963f
I20260809 14:30:17.210064 31942 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:17.210683 31942 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:17.211584 31942 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_02301a82edfd45bd9511edf923e1028d [id=c1935d3bc49c405abe4d006932820224]). Expected version 3 got 0
I20260809 14:30:17.185036 32691 tablet_bootstrap.cc:654] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:17.212100 31942 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:17.212635 31942 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:17.213131 31942 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:17.213593 31942 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:17.213855 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ff6ff27f6144d08d68a5e79c12963f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:17.218210 1048 raft_consensus.cc:482] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:17.218688 1048 raft_consensus.cc:504] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.221105 1048 leader_election.cc:288] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:17.226831 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ff6ff27f6144d08d68a5e79c12963f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:17.227929 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 197b996294454f22a98b37dab6bb2b1a (DEFAULT_TABLE table=default.Table_83c723c78d1b466f96b992e07866b7e0 [id=345fabe93b8b4d038d0faf2de4dc8556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.228931 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197b996294454f22a98b37dab6bb2b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.230038 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf222c49c934d52a2642f20e8575702" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:17.230880 32296 tablet_service.cc:1467] Processing CreateTablet for tablet e8e671bc84a94257b83dec0ce897e257 (DEFAULT_TABLE table=default.Table_c6a19ad0e6594d42833aa8edc7b517ad [id=2a218160937e463b8cb540104f4fee4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.231956 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e671bc84a94257b83dec0ce897e257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.234481 32477 tablet_service.cc:1467] Processing CreateTablet for tablet 18af64fecd2540d9a3ccac7bee39399e (DEFAULT_TABLE table=default.Table_222d7926360d490abc0717b23e7ff1f5 [id=6f6a067cb2a54359a9bfd2267b554581]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:17.241015 32443 leader_election.cc:341] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:17.242293 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 18af64fecd2540d9a3ccac7bee39399e (DEFAULT_TABLE table=default.Table_222d7926360d490abc0717b23e7ff1f5 [id=6f6a067cb2a54359a9bfd2267b554581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.243443 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18af64fecd2540d9a3ccac7bee39399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.236258 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 197b996294454f22a98b37dab6bb2b1a (DEFAULT_TABLE table=default.Table_83c723c78d1b466f96b992e07866b7e0 [id=345fabe93b8b4d038d0faf2de4dc8556]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:17.246371 31944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:51493 (request call id 50) took 1037 ms (client timeout 1000 ms). Trace:
W20260809 14:30:17.246752 31944 rpcz_store.cc:269] 0809 14:30:16.209375 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:16.209733 (+ 358us) service_pool.cc:227] Handling call
0809 14:30:17.246307 (+1036574us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":3}
I20260809 14:30:17.248157 32483 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cdfc56edcd34696a7b806b0ee34d594 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:17.249462 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_1474c6854a634c82909f0cc97490d2ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.250897 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1474c6854a634c82909f0cc97490d2ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.260541 996 ts_tablet_manager.cc:1905] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:17.261274 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf222c49c934d52a2642f20e8575702" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:17.262542 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 18af64fecd2540d9a3ccac7bee39399e (DEFAULT_TABLE table=default.Table_222d7926360d490abc0717b23e7ff1f5 [id=6f6a067cb2a54359a9bfd2267b554581]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:17.262851 32439 leader_election.cc:341] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:17.263231 32439 leader_election.cc:302] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:17.263514 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18af64fecd2540d9a3ccac7bee39399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.265072 32140 tablet_service.cc:1467] Processing CreateTablet for tablet e8e671bc84a94257b83dec0ce897e257 (DEFAULT_TABLE table=default.Table_c6a19ad0e6594d42833aa8edc7b517ad [id=2a218160937e463b8cb540104f4fee4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.265614 1048 raft_consensus.cc:2726] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:17.268328 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cb7842617a4779bf110e4ea254e011" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:17.269114 32508 raft_consensus.cc:3037] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:17.271157 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e671bc84a94257b83dec0ce897e257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.274976 32478 tablet_service.cc:1467] Processing CreateTablet for tablet 782ba14ca8bc41429a086d49cc40434f (DEFAULT_TABLE table=default.Table_dad01c31cb0242f9a295c92854814711 [id=fa449cee9fad4fd6a349b0d5a29a08ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.277036 32479 tablet_service.cc:1467] Processing CreateTablet for tablet 666bea53f14541368730f69402c2b411 (DEFAULT_TABLE table=default.Table_cc226a9bef8a4d6bb50d7bf50e57193c [id=65895a52fe1d497495bec9291b6a695d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.278600 32483 tablet_service.cc:1467] Processing CreateTablet for tablet e8e671bc84a94257b83dec0ce897e257 (DEFAULT_TABLE table=default.Table_c6a19ad0e6594d42833aa8edc7b517ad [id=2a218160937e463b8cb540104f4fee4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.293771 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:17.293367 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 782ba14ca8bc41429a086d49cc40434f (DEFAULT_TABLE table=default.Table_dad01c31cb0242f9a295c92854814711 [id=fa449cee9fad4fd6a349b0d5a29a08ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.294900 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 782ba14ca8bc41429a086d49cc40434f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.296792 31940 master_service.cc:946] ReplaceTablet: received request to replace tablet 9e38a13e6e1c480880eab4bc2d2113cd from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:17.280027 32508 raft_consensus.cc:2454] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:17.303972 32133 tablet_service.cc:1467] Processing CreateTablet for tablet 666bea53f14541368730f69402c2b411 (DEFAULT_TABLE table=default.Table_cc226a9bef8a4d6bb50d7bf50e57193c [id=65895a52fe1d497495bec9291b6a695d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.305097 32133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666bea53f14541368730f69402c2b411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.305145 32141 tablet_service.cc:1467] Processing CreateTablet for tablet 667d3ad942cb48a08c286d1ea602d000 (DEFAULT_TABLE table=default.Table_373053e6ca574ffa8cee29e912be22da [id=9f79f9809cea4c35bf91851b7294cbec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.306543 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 7ce3271ee8684662a9d3cd56157e572a (DEFAULT_TABLE table=default.Table_f13fa07f50084edc949a28f2c35bd624 [id=845c15ebc0e34c0e94c22ae92622df0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.267431 32134 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ff6ff27f6144d08d68a5e79c12963f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:17.307374 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667d3ad942cb48a08c286d1ea602d000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.310164 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 7ce3271ee8684662a9d3cd56157e572a (DEFAULT_TABLE table=default.Table_f13fa07f50084edc949a28f2c35bd624 [id=845c15ebc0e34c0e94c22ae92622df0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.311894 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 667d3ad942cb48a08c286d1ea602d000 (DEFAULT_TABLE table=default.Table_373053e6ca574ffa8cee29e912be22da [id=9f79f9809cea4c35bf91851b7294cbec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.313385 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 96a0005fc1394bf9a9fde19a2d2fbada (DEFAULT_TABLE table=default.Table_a737e7da4d6445b7b9e29bc4a9fb13aa [id=9effad5f48cc48e5a24bf2cd91f1d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.316180 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ce3271ee8684662a9d3cd56157e572a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.318549 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cb7842617a4779bf110e4ea254e011" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:17.320165 32164 raft_consensus.cc:3037] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:17.321811 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 96a0005fc1394bf9a9fde19a2d2fbada (DEFAULT_TABLE table=default.Table_a737e7da4d6445b7b9e29bc4a9fb13aa [id=9effad5f48cc48e5a24bf2cd91f1d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.322340 32134 tablet_service.cc:1467] Processing CreateTablet for tablet b84d242df9f44726a6771a0541e3a124 (DEFAULT_TABLE table=default.Table_f2894c35d51a436fb8ff7bc574624581 [id=65379b49137945acb43d46f22b12a128]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.323266 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b27d4b6a8d4d47388f293a343c5d11ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.325011 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b27d4b6a8d4d47388f293a343c5d11ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.325503 32134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84d242df9f44726a6771a0541e3a124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.327838 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a0005fc1394bf9a9fde19a2d2fbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.330576 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_93526ced6a324be680b48ad0d4587fe6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.331943 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93526ced6a324be680b48ad0d4587fe6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.336597 996 ts_tablet_manager.cc:1918] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:17.337008 996 log.cc:1196] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/e846e795144e48e3b2a1eca8c1b26489
I20260809 14:30:17.337723 996 ts_tablet_manager.cc:1939] T e846e795144e48e3b2a1eca8c1b26489 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:17.341743 32481 tablet_service.cc:1467] Processing CreateTablet for tablet b84d242df9f44726a6771a0541e3a124 (DEFAULT_TABLE table=default.Table_f2894c35d51a436fb8ff7bc574624581 [id=65379b49137945acb43d46f22b12a128]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.343384 32485 tablet_service.cc:1467] Processing CreateTablet for tablet ab045e730f3048a8b63b8962ea191ee7 (DEFAULT_TABLE table=default.Table_e6905fa41eb64a4c9eeabed102a72d23 [id=1544a31f65074713b012206c3634a8c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.344892 32484 tablet_service.cc:1467] Processing CreateTablet for tablet a4dca6591ac94fb59645db523e024a09 (DEFAULT_TABLE table=default.Table_a5839d042df748b5b52689e11f581dd6 [id=12ebe7f1d8a149b2a40db412119ef8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.346410 32139 tablet_service.cc:1467] Processing CreateTablet for tablet ab045e730f3048a8b63b8962ea191ee7 (DEFAULT_TABLE table=default.Table_e6905fa41eb64a4c9eeabed102a72d23 [id=1544a31f65074713b012206c3634a8c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.347380 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab045e730f3048a8b63b8962ea191ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.349573 32147 tablet_service.cc:1467] Processing CreateTablet for tablet a4dca6591ac94fb59645db523e024a09 (DEFAULT_TABLE table=default.Table_a5839d042df748b5b52689e11f581dd6 [id=12ebe7f1d8a149b2a40db412119ef8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.350441 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4dca6591ac94fb59645db523e024a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.371810 32489 tablet_service.cc:1467] Processing CreateTablet for tablet c423cc085ce9488697bb63159228541f (DEFAULT_TABLE table=default.Table_60f882b13b3d48f28134afe6dfed9f10 [id=4ff5e70bfad84d4b88d5966188c69279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.375893 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 782ba14ca8bc41429a086d49cc40434f (DEFAULT_TABLE table=default.Table_dad01c31cb0242f9a295c92854814711 [id=fa449cee9fad4fd6a349b0d5a29a08ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.377051 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 782ba14ca8bc41429a086d49cc40434f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.381278 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:17.382088 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 5121e9f5a85a45d285d78f41a442769a (table default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba]). Expected version 1 got 0
I20260809 14:30:17.382009 32298 tablet_service.cc:1467] Processing CreateTablet for tablet b84d242df9f44726a6771a0541e3a124 (DEFAULT_TABLE table=default.Table_f2894c35d51a436fb8ff7bc574624581 [id=65379b49137945acb43d46f22b12a128]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.382712 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:17.383139 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84d242df9f44726a6771a0541e3a124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.383917 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:17.389381 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 666bea53f14541368730f69402c2b411 (DEFAULT_TABLE table=default.Table_cc226a9bef8a4d6bb50d7bf50e57193c [id=65895a52fe1d497495bec9291b6a695d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.390596 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666bea53f14541368730f69402c2b411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.393895 32304 tablet_service.cc:1467] Processing CreateTablet for tablet 96a0005fc1394bf9a9fde19a2d2fbada (DEFAULT_TABLE table=default.Table_a737e7da4d6445b7b9e29bc4a9fb13aa [id=9effad5f48cc48e5a24bf2cd91f1d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.395412 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a0005fc1394bf9a9fde19a2d2fbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.396746 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 667d3ad942cb48a08c286d1ea602d000 (DEFAULT_TABLE table=default.Table_373053e6ca574ffa8cee29e912be22da [id=9f79f9809cea4c35bf91851b7294cbec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.397562 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667d3ad942cb48a08c286d1ea602d000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.401338 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 7ce3271ee8684662a9d3cd56157e572a (DEFAULT_TABLE table=default.Table_f13fa07f50084edc949a28f2c35bd624 [id=845c15ebc0e34c0e94c22ae92622df0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.402510 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ce3271ee8684662a9d3cd56157e572a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.412421 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 9e38a13e6e1c480880eab4bc2d2113cd
I20260809 14:30:17.413223 32295 tablet_service.cc:1467] Processing CreateTablet for tablet ab045e730f3048a8b63b8962ea191ee7 (DEFAULT_TABLE table=default.Table_e6905fa41eb64a4c9eeabed102a72d23 [id=1544a31f65074713b012206c3634a8c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.415269 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab045e730f3048a8b63b8962ea191ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.416538 31940 catalog_manager.cc:6565] ReplaceTablet: tablet 9e38a13e6e1c480880eab4bc2d2113cd deleted and replaced by tablet 36350e37e5834491a352417543f7564c
I20260809 14:30:17.419571 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e38a13e6e1c480880eab4bc2d2113cd with delete_type TABLET_DATA_DELETED (replaced by tablet 36350e37e5834491a352417543f7564c) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:17.419764 32300 tablet_service.cc:1467] Processing CreateTablet for tablet a4dca6591ac94fb59645db523e024a09 (DEFAULT_TABLE table=default.Table_a5839d042df748b5b52689e11f581dd6 [id=12ebe7f1d8a149b2a40db412119ef8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.424193 32303 tablet_service.cc:1467] Processing CreateTablet for tablet c423cc085ce9488697bb63159228541f (DEFAULT_TABLE table=default.Table_60f882b13b3d48f28134afe6dfed9f10 [id=4ff5e70bfad84d4b88d5966188c69279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.429164 32305 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e38a13e6e1c480880eab4bc2d2113cd with delete_type TABLET_DATA_DELETED (replaced by tablet 36350e37e5834491a352417543f7564c) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:17.454376 32303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c423cc085ce9488697bb63159228541f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.459415 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4dca6591ac94fb59645db523e024a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.476809 1048 raft_consensus.cc:482] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:17.477471 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_895a6d5bcb404eb8aaec0484b109e6aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:17.477833 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:17.477505 1048 raft_consensus.cc:504] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:17.479035 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_895a6d5bcb404eb8aaec0484b109e6aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.479096 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 8b53f7d398d64701b4504db5c9c440b8 (table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]). Expected version 2 got 0
I20260809 14:30:17.480387 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:17.481333 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_18d2bdbeb7ad42e9b0edf59e2e89e8b3" } new_table_name: "default.Table_4f08a6e32ba345a2871a434f4fef3818" modify_external_catalogs: true
I20260809 14:30:17.483556 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_02301a82edfd45bd9511edf923e1028d" } modify_external_catalogs: true
I20260809 14:30:17.483805 1048 leader_election.cc:288] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:17.484764 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_02301a82edfd45bd9511edf923e1028d" } modify_external_catalogs: true
I20260809 14:30:17.498988 32705 ts_tablet_manager.cc:1428] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.481s user 0.007s sys 0.000s
I20260809 14:30:17.506388 1020 ts_tablet_manager.cc:1918] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:17.516546 1020 log.cc:1196] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/9922cad6d10945d5b013a168a3496676
I20260809 14:30:17.519407 1020 ts_tablet_manager.cc:1939] T 9922cad6d10945d5b013a168a3496676 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:17.482005 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_02301a82edfd45bd9511edf923e1028d [id=c1935d3bc49c405abe4d006932820224]). Expected version 3 got 0
I20260809 14:30:17.533279 31945 catalog_manager.cc:5495] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:17.534965 31945 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 239ad14cb3a642e08bace0759bc0b732 (table default.Table_e7dbc14f657745d390a15586fda27677 [id=284b10d1269c44dea39dc24aade3459c]). Expected version 1 got 0
W20260809 14:30:17.537209 1074 log.cc:924] Time spent T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b: Append to log took a long time: real 0.356s user 0.000s sys 0.000s
I20260809 14:30:17.538177 32693 tablet_bootstrap.cc:654] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:17.542503 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_348f7a78a51f40369f9fbd23580ff6b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:17.544889 32691 tablet_bootstrap.cc:492] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
W20260809 14:30:17.543475 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b80e4ff9e3742159be8a78c633c0f56 consensus state has no leader
I20260809 14:30:17.545400 32691 ts_tablet_manager.cc:1397] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.463s user 0.033s sys 0.082s
W20260809 14:30:17.545012 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_348f7a78a51f40369f9fbd23580ff6b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.554122 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18af64fecd2540d9a3ccac7bee39399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.557799 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197b996294454f22a98b37dab6bb2b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.561151 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 782ba14ca8bc41429a086d49cc40434f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.565178 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666bea53f14541368730f69402c2b411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.569000 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e671bc84a94257b83dec0ce897e257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.573030 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ce3271ee8684662a9d3cd56157e572a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.576952 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667d3ad942cb48a08c286d1ea602d000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.582017 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a0005fc1394bf9a9fde19a2d2fbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.585374 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84d242df9f44726a6771a0541e3a124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.586529 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab045e730f3048a8b63b8962ea191ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.588004 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4dca6591ac94fb59645db523e024a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.588900 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c423cc085ce9488697bb63159228541f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.589573 32691 raft_consensus.cc:348] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.590240 32691 raft_consensus.cc:374] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:17.590468 32691 raft_consensus.cc:729] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:30:17.639179 31735 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 365153us
I20260809 14:30:17.642812 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:17.658790 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet d885baba6d6045e8b1ebcab6513719bb (table default.Table_7e7ed61d048442e7902bbb1f79677b20 [id=98bb45c719374b96bcb770bdf44f5553]). Expected version 1 got 0
I20260809 14:30:17.659706 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 88644048fa4a443fac09e951d78b3063 (table default.Table_569de184b14243abb58d1414941be5aa [id=67767d6db83f4ae6ab265b854d2ad695]). Expected version 2 got 0
I20260809 14:30:17.661461 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:17.662235 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:17.663199 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 32cb7842617a4779bf110e4ea254e011 (table default.Table_b86f8f5a1acc4bacb6b93b4acd413b0f [id=2d493ef1e83d47da9117ecd5312dacdc]). Expected version 1 got 0
I20260809 14:30:17.667019 31937 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_a3796b4fbd05463eb6193485302bcd5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.672921 31937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3796b4fbd05463eb6193485302bcd5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.671124 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_fe15193782d744c19d096d034729510a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.677721 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe15193782d744c19d096d034729510a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.591116 32691 consensus_queue.cc:260] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.690665 32691 ts_tablet_manager.cc:1428] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.145s user 0.003s sys 0.007s
I20260809 14:30:17.691577 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:17.710837 32691 tablet_bootstrap.cc:492] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:17.724900 32705 tablet_bootstrap.cc:492] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:17.731724 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5b7be37c56a740b894577b6dece2d3e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.735293 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b7be37c56a740b894577b6dece2d3e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.737317 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 239ad14cb3a642e08bace0759bc0b732 (table default.Table_e7dbc14f657745d390a15586fda27677 [id=284b10d1269c44dea39dc24aade3459c]). Expected version 1 got 0
I20260809 14:30:17.741657 31938 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 67afcb47ffc947c5b1d98c96de333116 (table default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3]). Expected version 2 got 0
I20260809 14:30:17.746460 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet e74db9caa7f94059b1bba4a082fe57ef
I20260809 14:30:17.747993 31943 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:49782 (request call id 324) took 2423 ms. Trace:
I20260809 14:30:17.749097 31946 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:49782 (request call id 331) took 1487 ms. Trace:
I20260809 14:30:17.748311 31943 rpcz_store.cc:276] 0809 14:30:15.324893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.970735 (+645842us) service_pool.cc:227] Handling call
0809 14:30:15.982346 (+ 11611us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d9466 00007ffa4b01ab34 00007ffa4b01aac2 00007ffa4b01a9f1 00007ffa4b01a96c 00007ffa4b019c22 00007ffa492b894c 00007ffa492b7d86
0809 14:30:16.020200 (+ 37854us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:16.020849 (+ 649us) catalog_manager.cc:2752] Modifying in-memory table state
0809 14:30:16.021118 (+ 269us) catalog_manager.cc:2761] Locking tablets
0809 14:30:16.021817 (+ 699us) catalog_manager.cc:2775] Removing table and tablets from system table
0809 14:30:16.111465 (+ 89648us) catalog_manager.cc:2793] Removing table from by-name map
0809 14:30:16.111762 (+ 297us) catalog_manager.cc:2809] Committing in-memory state
0809 14:30:17.746365 (+1634603us) catalog_manager.cc:2813] Aborting table tasks
0809 14:30:17.747925 (+ 1560us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:16.023823 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:16.024274 (+ 451us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:16.024303 (+ 29us) write_op.cc:435] Acquired schema lock
0809 14:30:16.024320 (+ 17us) tablet.cc:660] Decoding operations
0809 14:30:16.024738 (+ 418us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:16.024792 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:16.024817 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0809 14:30:16.025223 (+ 406us) tablet.cc:699] Row locks acquired
0809 14:30:16.025244 (+ 21us) write_op.cc:260] PREPARE: finished
0809 14:30:16.025360 (+ 116us) write_op.cc:270] Start()
0809 14:30:16.025454 (+ 94us) write_op.cc:276] Timestamp: P: 1786285816025281 usec, L: 0
0809 14:30:16.025477 (+ 23us) op_driver.cc:331] REPLICATION: starting
0809 14:30:16.026005 (+ 528us) log.cc:842] Serialized 762 byte log entry
0809 14:30:16.047266 (+ 21261us) op_driver.cc:444] REPLICATION: finished
0809 14:30:16.048018 (+ 752us) write_op.cc:301] APPLY: starting
0809 14:30:16.048076 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:16.048181 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:16.048308 (+ 127us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:16.048665 (+ 357us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:16.048819 (+ 154us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 14:30:16.048862 (+ 43us) write_op.cc:312] APPLY: finished
0809 14:30:16.049031 (+ 169us) log.cc:842] Serialized 29 byte log entry
0809 14:30:16.049170 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:16.049238 (+ 68us) write_op.cc:454] Released schema lock
0809 14:30:16.049268 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:16.049433 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:16.049465 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":24950528,"child_traces":[["op",{"apply.queue_time_us":605,"apply.run_cpu_time_us":1618,"apply.run_wall_time_us":1618,"num_ops":2,"prepare.queue_time_us":83,"prepare.run_cpu_time_us":2393,"prepare.run_wall_time_us":2392,"replication_time_us":21693}]]}
I20260809 14:30:17.749426 31946 rpcz_store.cc:276] 0809 14:30:16.261739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:16.261840 (+ 101us) service_pool.cc:227] Handling call
0809 14:30:16.262733 (+ 893us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0809 14:30:16.263447 (+ 714us) catalog_manager.cc:3616] Apply alter schema
0809 14:30:16.264041 (+ 594us) catalog_manager.cc:3639] Acquired catalog manager lock
0809 14:30:16.264265 (+ 224us) catalog_manager.cc:3786] Updating metadata on disk
0809 14:30:16.291326 (+ 27061us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:16.291349 (+ 23us) write_op.cc:454] Released schema lock
0809 14:30:16.291518 (+ 169us) catalog_manager.cc:3817] Committing alterations to in-memory state
0809 14:30:17.749042 (+1457524us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:16.265557 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:16.266004 (+ 447us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:16.266035 (+ 31us) write_op.cc:435] Acquired schema lock
0809 14:30:16.266053 (+ 18us) tablet.cc:660] Decoding operations
0809 14:30:16.266349 (+ 296us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:16.266397 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:16.266423 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:16.266585 (+ 162us) tablet.cc:699] Row locks acquired
0809 14:30:16.266605 (+ 20us) write_op.cc:260] PREPARE: finished
0809 14:30:16.266729 (+ 124us) write_op.cc:270] Start()
0809 14:30:16.266822 (+ 93us) write_op.cc:276] Timestamp: P: 1786285816266675 usec, L: 0
0809 14:30:16.266842 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 14:30:16.267374 (+ 532us) log.cc:842] Serialized 405 byte log entry
0809 14:30:16.279507 (+ 12133us) op_driver.cc:444] REPLICATION: finished
0809 14:30:16.279789 (+ 282us) write_op.cc:301] APPLY: starting
0809 14:30:16.279834 (+ 45us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:16.279922 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:16.280012 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:16.280162 (+ 150us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:16.280283 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:16.280322 (+ 39us) write_op.cc:312] APPLY: finished
0809 14:30:16.280495 (+ 173us) log.cc:842] Serialized 23 byte log entry
0809 14:30:16.280614 (+ 119us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:16.280671 (+ 57us) write_op.cc:454] Released schema lock
0809 14:30:16.280696 (+ 25us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":2414,"child_traces":[["op",{"apply.queue_time_us":154,"apply.run_cpu_time_us":1061,"apply.run_wall_time_us":1060,"num_ops":1,"prepare.queue_time_us":96,"prepare.run_cpu_time_us":2098,"prepare.run_wall_time_us":2098,"raft.queue_time_us":1183,"raft.run_cpu_time_us":590,"raft.run_wall_time_us":590,"replication_time_us":12593}]]}
I20260809 14:30:17.755563 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet e74db9caa7f94059b1bba4a082fe57ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:17.529101 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273ca6176a184a719538a9455afded4d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:17.769933 32140 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e38a13e6e1c480880eab4bc2d2113cd with delete_type TABLET_DATA_DELETED (replaced by tablet 36350e37e5834491a352417543f7564c) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:17.770954 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 496830us
I20260809 14:30:17.771628 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 5ac7afd531a746cab7618ead98b4c4d9 (table default.Table_f13ec0ce721048f4aabef02eb1f5cc44 [id=617231ca8be54c389a6c0d821b081eb2]). Expected version 1 got 0
I20260809 14:30:17.771930 32260 leader_election.cc:302] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:17.773011 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 239ad14cb3a642e08bace0759bc0b732 (table default.Table_e7dbc14f657745d390a15586fda27677 [id=284b10d1269c44dea39dc24aade3459c]). Expected version 1 got 0
I20260809 14:30:17.774621 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet ebc6ebda27b44048b5734e8798a980c2 (table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]). Expected version 1 got 0
I20260809 14:30:17.775532 31939 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:17.776468 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 2dc8d0cad33b4778af9fe9247743c38f
I20260809 14:30:17.778076 1020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 246) took 1790 ms. Trace:
I20260809 14:30:17.778467 1020 rpcz_store.cc:276] 0809 14:30:15.987229 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.987340 (+ 111us) service_pool.cc:227] Handling call
0809 14:30:16.834489 (+847149us) ts_tablet_manager.cc:1145] Deleting tablet 9922cad6d10945d5b013a168a3496676
0809 14:30:16.834533 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:16.978902 (+144369us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:17.506065 (+527163us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:17.777998 (+271933us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":279100,"tablet-delete.queue_time_us":846644}
I20260809 14:30:17.782620 1022 raft_consensus.cc:2781] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:17.783039 1022 raft_consensus.cc:686] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:17.770588 1009 raft_consensus.cc:686] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:17.790066 1009 consensus_queue.cc:237] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:17.793674 31948 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 75) took 2103 ms (client timeout 1000 ms). Trace:
W20260809 14:30:17.794063 31948 rpcz_store.cc:269] 0809 14:30:15.690207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.938367 (+248160us) service_pool.cc:227] Handling call
0809 14:30:17.793607 (+1855240us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":14}
W20260809 14:30:17.796903 31942 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.65:36333 (request call id 76) took 983 ms (client timeout 1000 ms). Trace:
W20260809 14:30:17.797677 31942 rpcz_store.cc:269] 0809 14:30:16.813868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:16.814013 (+ 145us) service_pool.cc:227] Handling call
0809 14:30:16.840446 (+ 26433us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b54000118d4. stack: 00007ffa44582033 00007ffa44581f30 00007ffa44103534 00000000004db2b4 00000000004db249 00007ffa4b48eaf0 00007ffa4b511f2a 00007ffa4d3ddd6c 00007ffa4d4d7c3e 00007ffa4b00efb4 00007ffa4b00ef42 00007ffa4b00ee71 00007ffa4b00edec 00007ffa4b00e0a2 00007ffa492b894c 00007ffa492b7d86
0809 14:30:17.796843 (+956397us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":42653,"reported_tablets":15,"spinlock_wait_cycles":53861248}
I20260809 14:30:17.783802 1022 consensus_queue.cc:237] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:17.780360 1020 tablet_replica.cc:330] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:17.806257 1020 raft_consensus.cc:2229] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:17.806861 1020 raft_consensus.cc:2258] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:17.808981 1020 ts_tablet_manager.cc:1905] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:17.835774 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_513f49ccff6d4f15bd52a9814a93e513" } new_table_name: "default.Table_caf2a08a3d964393a1d28f6266b068fe" modify_external_catalogs: true
I20260809 14:30:17.840029 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_f01ac5f2b8b54a57a444f006d814b132"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.842481 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f01ac5f2b8b54a57a444f006d814b132 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.853014 32143 tablet_service.cc:1467] Processing CreateTablet for tablet c423cc085ce9488697bb63159228541f (DEFAULT_TABLE table=default.Table_60f882b13b3d48f28134afe6dfed9f10 [id=4ff5e70bfad84d4b88d5966188c69279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:17.854029 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fdaef8a95e4aa0b23018d384442d16 with delete_type TABLET_DATA_DELETED (replaced by tablet 1eb9d7c27381473aa87b718cca57b678) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:17.854206 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c423cc085ce9488697bb63159228541f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:17.854894 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f171260cb764ac284aef85e4219767e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:17.855536 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad3946f5de4387af3627a88b6404cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:01 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:17.865483 31945 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 59) took 2201 ms (client timeout 1000 ms). Trace:
I20260809 14:30:17.874862 1084 raft_consensus.cc:482] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:30:17.865844 31945 rpcz_store.cc:269] 0809 14:30:15.664412 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.848987 (+184575us) service_pool.cc:227] Handling call
0809 14:30:17.865400 (+2016413us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:17.581945 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:17.582283 (+ 338us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:17.582306 (+ 23us) write_op.cc:435] Acquired schema lock
0809 14:30:17.582317 (+ 11us) tablet.cc:660] Decoding operations
0809 14:30:17.582545 (+ 228us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:17.582582 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:17.582602 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:17.582731 (+ 129us) tablet.cc:699] Row locks acquired
0809 14:30:17.582743 (+ 12us) write_op.cc:260] PREPARE: finished
0809 14:30:17.582840 (+ 97us) write_op.cc:270] Start()
0809 14:30:17.582909 (+ 69us) write_op.cc:276] Timestamp: P: 1786285817582782 usec, L: 0
0809 14:30:17.582926 (+ 17us) op_driver.cc:331] REPLICATION: starting
0809 14:30:17.583454 (+ 528us) log.cc:842] Serialized 533 byte log entry
0809 14:30:17.592196 (+ 8742us) op_driver.cc:444] REPLICATION: finished
0809 14:30:17.592705 (+ 509us) write_op.cc:301] APPLY: starting
0809 14:30:17.592767 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:17.592893 (+ 126us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:17.593004 (+ 111us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:17.593281 (+ 277us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:17.593494 (+ 213us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:17.593588 (+ 94us) write_op.cc:312] APPLY: finished
0809 14:30:17.593983 (+ 395us) log.cc:842] Serialized 23 byte log entry
0809 14:30:17.594207 (+ 224us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:17.594298 (+ 91us) write_op.cc:454] Released schema lock
0809 14:30:17.594341 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:17.594619 (+ 278us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:17.594692 (+ 73us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":303,"apply.run_cpu_time_us":2208,"apply.run_wall_time_us":2206,"num_ops":1,"prepare.queue_time_us":94,"prepare.run_cpu_time_us":1752,"prepare.run_wall_time_us":1752,"raft.queue_time_us":1154,"raft.run_cpu_time_us":1100,"raft.run_wall_time_us":1139,"replication_time_us":9189}]]}
W20260809 14:30:17.887130 31932 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 37a3a71874fd44bf865b11c05e84ca84 (table default.Table_61b1d3ac3b1f4314b31339c66858eb78 [id=2cf0c0e1fb6f40c29bb391b38988e91a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 37a3a71874fd44bf865b11c05e84ca84 consensus state has no leader
I20260809 14:30:17.896934 31939 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_3f10c680e2bc43569707615d0eb1dd66 [id=7e9823d249dd4c4ab3362be3b7a331b3] alter complete (version 2)
I20260809 14:30:17.682169 32165 raft_consensus.cc:2454] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 2.
I20260809 14:30:17.905022 32140 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc8d0cad33b4778af9fe9247743c38f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:17 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:17.910481 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1a42b931475a407c971911e55d562d70 consensus state has no leader
I20260809 14:30:17.936247 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9f220d0bb5564519b3d75cc03a8ec042"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:17.938894 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_153b78d9a7934c7ca382c10459415af0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:17.957309 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f220d0bb5564519b3d75cc03a8ec042 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.958981 1086 raft_consensus.cc:482] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:17.959496 1086 raft_consensus.cc:504] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:17.960568 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_153b78d9a7934c7ca382c10459415af0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:17.960611 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 250) took 1872 ms. Trace:
I20260809 14:30:17.961133 996 rpcz_store.cc:276] 0809 14:30:16.088548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:16.088792 (+ 244us) service_pool.cc:227] Handling call
0809 14:30:17.092606 (+1003814us) ts_tablet_manager.cc:1145] Deleting tablet e846e795144e48e3b2a1eca8c1b26489
0809 14:30:17.096636 (+ 4030us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:17.308882 (+212246us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:17.336334 (+ 27452us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:17.950286 (+613952us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":967813}
I20260809 14:30:17.963131 996 tablet_replica.cc:330] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:17.968959 31938 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1] alter complete (version 2)
I20260809 14:30:17.978629 32481 tablet_service.cc:1514] Processing DeleteTablet for tablet e74db9caa7f94059b1bba4a082fe57ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:17.979378 32691 tablet_bootstrap.cc:654] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:17.981105 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet e846e795144e48e3b2a1eca8c1b26489 (table default.Table_f72f5e4a712b4a519089aa13f2060a2a [id=90072fc6ca834e5a8ab518ccd2131c84]) successfully deleted
I20260809 14:30:17.958591 32132 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:18.014235 1004 ts_tablet_manager.cc:1918] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:18.014824 1004 log.cc:1196] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/e846e795144e48e3b2a1eca8c1b26489
I20260809 14:30:18.021476 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_5b7be37c56a740b894577b6dece2d3e4" } new_table_name: "default.Table_cdb56a7b91ec4bf2ad8f560243ecb761" modify_external_catalogs: true
I20260809 14:30:18.026786 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_d1d760ad86f94839bf3b5b71dd8809aa" } modify_external_catalogs: true
I20260809 14:30:18.027422 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_d1d760ad86f94839bf3b5b71dd8809aa" } modify_external_catalogs: true
I20260809 14:30:18.032856 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b46cfe02c4084ffbb8825b06376fc849" } modify_external_catalogs: true
I20260809 14:30:18.033417 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b46cfe02c4084ffbb8825b06376fc849" } modify_external_catalogs: true
W20260809 14:30:18.003054 32164 consensus_meta.cc:287] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.682s user 0.006s sys 0.001s
I20260809 14:30:18.046142 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_3344e8d1d6c347aaa907473e456adb3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.047755 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3344e8d1d6c347aaa907473e456adb3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.015826 1004 ts_tablet_manager.cc:1939] T e846e795144e48e3b2a1eca8c1b26489 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:18.058576 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 254) took 1968 ms. Trace:
I20260809 14:30:18.059118 1004 rpcz_store.cc:276] 0809 14:30:16.089689 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:16.089841 (+ 152us) service_pool.cc:227] Handling call
0809 14:30:16.550419 (+460578us) ts_tablet_manager.cc:1145] Deleting tablet e846e795144e48e3b2a1eca8c1b26489
0809 14:30:16.550473 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:16.950824 (+400351us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:18.013923 (+1063099us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:18.058489 (+ 44566us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":49628,"tablet-delete.queue_time_us":459878}
I20260809 14:30:18.085091 31938 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_aaad0560a0364aad9a26bbbcecc64736 [id=94d067b57a6d4386be14f8b3f4cb1aba] alter complete (version 1)
I20260809 14:30:18.094611 32485 tablet_service.cc:1514] Processing DeleteTablet for tablet 301b53eeb3be41a2b8ff2598d1b531d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:07 UTC) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:18.101966 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8ecf7edec2b48dcbaea4f1413b44f95 has no consensus state
I20260809 14:30:18.104296 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet c468382a28934d50babf7bbc9a834c08
I20260809 14:30:18.113147 32705 tablet_bootstrap.cc:654] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:18.113041 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet c468382a28934d50babf7bbc9a834c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:18.116698 31938 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 60) took 1123 ms (client timeout 1000 ms). Trace:
I20260809 14:30:18.118430 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet e846e795144e48e3b2a1eca8c1b26489 (table default.Table_f72f5e4a712b4a519089aa13f2060a2a [id=90072fc6ca834e5a8ab518ccd2131c84]) successfully deleted
W20260809 14:30:18.131341 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 94ff6ff27f6144d08d68a5e79c12963f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94ff6ff27f6144d08d68a5e79c12963f already in progress: creating tablet
I20260809 14:30:18.143079 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_61b1d3ac3b1f4314b31339c66858eb78" } new_table_name: "default.Table_60947825f97f4a7981e3301069cbd4ca" modify_external_catalogs: true
I20260809 14:30:18.155258 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet faf222c49c934d52a2642f20e8575702 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:04 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.163378 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 273ca6176a184a719538a9455afded4d with delete_type TABLET_DATA_DELETED (replaced by tablet 8c06535e3b5e441c89e9db7431267f98) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:18.165019 1001 lock_manager.cc:586] Waited 3 seconds to obtain row lock on key '<redacted>' (key hash 1495769309838729370) tablet 00000000000000000000000000000000 cur holder 0x7b5800060900
I20260809 14:30:17.993042 1082 raft_consensus.cc:482] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:18.172971 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_8405c747a7ce4e579d0fe4c17912d70b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.174732 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8405c747a7ce4e579d0fe4c17912d70b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.175945 1004 tablet_replica.cc:330] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:18.176801 1004 raft_consensus.cc:2229] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:18.177289 1004 raft_consensus.cc:2258] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:18.179330 1004 ts_tablet_manager.cc:1905] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:18.179483 32477 consensus_meta.cc:287] T 18af64fecd2540d9a3ccac7bee39399e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.586s user 0.000s sys 0.007s
I20260809 14:30:18.181152 996 raft_consensus.cc:2229] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:18.181790 996 raft_consensus.cc:2258] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:18.184010 996 ts_tablet_manager.cc:1905] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:18.185199 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 8a72259ef0d542e3a9bcb33e3da2cec9 (DEFAULT_TABLE table=default.Table_85987fca61094acbb4a924e9ceeec96a [id=d1d5be11f46c45cb8a1ded80c637e3a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.186170 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a72259ef0d542e3a9bcb33e3da2cec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.186224 32132 tablet_service.cc:1467] Processing CreateTablet for tablet 0b476fe571374e33a3b4d11057c713ea (DEFAULT_TABLE table=default.Table_8eb42cdceb174196b7743f3d3d12edae [id=9cfa749bc1fa41d89222d651922c7a8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.189819 1086 leader_election.cc:288] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:18.190486 32132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b476fe571374e33a3b4d11057c713ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:18.190860 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 37a3a71874fd44bf865b11c05e84ca84 consensus state has no leader
I20260809 14:30:17.931342 1084 raft_consensus.cc:504] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:18.196077 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 43931b0436c4472eb12ee1afd46c1900 (DEFAULT_TABLE table=default.Table_e6381d3c953a4958ada0a38267050cc0 [id=03607042808746c19f9e8a26eb7fa135]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.197074 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43931b0436c4472eb12ee1afd46c1900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.204835 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_82a85f2bf88341b0aa7081c4484a2817"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.172238 32303 consensus_meta.cc:287] T c423cc085ce9488697bb63159228541f P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.503s user 0.008s sys 0.002s
W20260809 14:30:18.206542 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82a85f2bf88341b0aa7081c4484a2817 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:18.205494 32693 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.663s user 0.014s sys 0.032s
I20260809 14:30:18.211218 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 9922cad6d10945d5b013a168a3496676 (table default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b]) successfully deleted
W20260809 14:30:18.212296 32693 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.670s user 0.015s sys 0.032s
I20260809 14:30:18.213992 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet c468382a28934d50babf7bbc9a834c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.214102 32489 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bad186f0ccf4c15a4f0b16bfffa945c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:13 UTC) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:18.206789 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 434900us
I20260809 14:30:18.217432 1084 leader_election.cc:288] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:18.214845 32439 leader_election.cc:341] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:18.241897 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet e74db9caa7f94059b1bba4a082fe57ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e74db9caa7f94059b1bba4a082fe57ef already in progress: creating tablet
W20260809 14:30:18.242755 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 9e38a13e6e1c480880eab4bc2d2113cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9e38a13e6e1c480880eab4bc2d2113cd already in progress: creating tablet
W20260809 14:30:18.249927 32145 consensus_meta.cc:287] T 96a0005fc1394bf9a9fde19a2d2fbada P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.508s user 0.008s sys 0.000s
I20260809 14:30:18.253199 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_6d21f9a0c69445bfb2882750ba0792dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.254585 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d21f9a0c69445bfb2882750ba0792dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.272125 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 5ad5bf5ed5cc4c66a4c87fc21209c555
I20260809 14:30:18.283982 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 514327321bf142379e8103a4b8c1a937 (DEFAULT_TABLE table=default.Table_4eccea50c485475c9e4a6f9e9990cac1 [id=5aef0a1444ad4cba9e08edb834962018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.312121 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e7dbc14f657745d390a15586fda27677" } modify_external_catalogs: true
I20260809 14:30:18.312665 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_e7dbc14f657745d390a15586fda27677" } modify_external_catalogs: true
W20260809 14:30:18.313344 1099 log.cc:924] Time spent T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260809 14:30:18.315491 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 9bd2360a3974492994a9a5b1ce8d2937 (DEFAULT_TABLE table=default.Table_b341959508524bd796f68f68e26f6c82 [id=7a0e2e1cd3e342de8bad2f981635a828]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.316684 32146 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:18.317225 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bd2360a3974492994a9a5b1ce8d2937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.317094 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:18.322322 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_9f220d0bb5564519b3d75cc03a8ec042" } new_table_name: "default.Table_089b0a96a71948bd8bc0df4624d303f2" modify_external_catalogs: true
I20260809 14:30:18.326809 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb90db2df534a3dbca8e5fa643eebb7 (DEFAULT_TABLE table=default.Table_5a11d072d3c94cccb6823afc0b64eb83 [id=a0fbc8aa31aa4719a6c5d04575b5c5fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.349918 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 239ad14cb3a642e08bace0759bc0b732
I20260809 14:30:18.336491 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 7ae3dd52b60842a083ae2609d0017b69 (DEFAULT_TABLE table=default.Table_e6685adbd7a7408f9fc60d67696e3b04 [id=33608e9192854140a2022cb0b0aa92f6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:18.362326 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 060e3ed3be3140559a59f7204ce0a2b6 has no consensus state
W20260809 14:30:18.117125 31938 rpcz_store.cc:269] 0809 14:30:16.993253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.045526 (+ 52273us) service_pool.cc:227] Handling call
0809 14:30:18.116446 (+1070920us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:17.953492 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:17.953883 (+ 391us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:17.953910 (+ 27us) write_op.cc:435] Acquired schema lock
0809 14:30:17.953930 (+ 20us) tablet.cc:660] Decoding operations
0809 14:30:17.954243 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:17.954282 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:17.954305 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:17.954456 (+ 151us) tablet.cc:699] Row locks acquired
0809 14:30:17.954484 (+ 28us) write_op.cc:260] PREPARE: finished
0809 14:30:17.954592 (+ 108us) write_op.cc:270] Start()
0809 14:30:17.954679 (+ 87us) write_op.cc:276] Timestamp: P: 1786285817954553 usec, L: 0
0809 14:30:17.954700 (+ 21us) op_driver.cc:331] REPLICATION: starting
0809 14:30:17.955226 (+ 526us) log.cc:842] Serialized 356 byte log entry
0809 14:30:17.966021 (+ 10795us) op_driver.cc:444] REPLICATION: finished
0809 14:30:17.966574 (+ 553us) write_op.cc:301] APPLY: starting
0809 14:30:17.966637 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:17.966818 (+ 181us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:17.967029 (+ 211us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:17.967317 (+ 288us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:17.967511 (+ 194us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:17.967574 (+ 63us) write_op.cc:312] APPLY: finished
0809 14:30:17.967791 (+ 217us) log.cc:842] Serialized 23 byte log entry
0809 14:30:17.968481 (+ 690us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:17.968572 (+ 91us) write_op.cc:454] Released schema lock
0809 14:30:17.968611 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:17.968850 (+ 239us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:17.968921 (+ 71us) write_op.cc:454] Released schema lock
Related trace 'op':
0809 14:30:18.025826 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 14:30:18.026166 (+ 340us) write_op.cc:432] Acquiring schema lock in shared mode
0809 14:30:18.026193 (+ 27us) write_op.cc:435] Acquired schema lock
0809 14:30:18.026204 (+ 11us) tablet.cc:660] Decoding operations
0809 14:30:18.026442 (+ 238us) write_op.cc:620] Acquiring the partition lock for write op
0809 14:30:18.026476 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0809 14:30:18.026495 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0809 14:30:18.026631 (+ 136us) tablet.cc:699] Row locks acquired
0809 14:30:18.026648 (+ 17us) write_op.cc:260] PREPARE: finished
0809 14:30:18.026758 (+ 110us) write_op.cc:270] Start()
0809 14:30:18.026836 (+ 78us) write_op.cc:276] Timestamp: P: 1786285818026713 usec, L: 0
0809 14:30:18.026852 (+ 16us) op_driver.cc:331] REPLICATION: starting
0809 14:30:18.027296 (+ 444us) log.cc:842] Serialized 356 byte log entry
0809 14:30:18.082935 (+ 55639us) op_driver.cc:444] REPLICATION: finished
0809 14:30:18.083634 (+ 699us) write_op.cc:301] APPLY: starting
0809 14:30:18.083707 (+ 73us) tablet.cc:1365] starting BulkCheckPresence
0809 14:30:18.083797 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0809 14:30:18.083912 (+ 115us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 14:30:18.084093 (+ 181us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 14:30:18.084243 (+ 150us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 14:30:18.084285 (+ 42us) write_op.cc:312] APPLY: finished
0809 14:30:18.084501 (+ 216us) log.cc:842] Serialized 23 byte log entry
0809 14:30:18.084659 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:18.084719 (+ 60us) write_op.cc:454] Released schema lock
0809 14:30:18.084749 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0809 14:30:18.084940 (+ 191us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:18.084963 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":340,"apply.run_cpu_time_us":2138,"apply.run_wall_time_us":2592,"num_ops":1,"prepare.queue_time_us":67514,"prepare.run_cpu_time_us":2088,"prepare.run_wall_time_us":23960,"raft.queue_time_us":106,"raft.run_cpu_time_us":1014,"raft.run_wall_time_us":3012,"replication_time_us":11244}],["op",{"apply.queue_time_us":539,"apply.run_cpu_time_us":1488,"apply.run_wall_time_us":1486,"num_ops":1,"prepare.queue_time_us":53895,"prepare.run_cpu_time_us":1649,"prepare.run_wall_time_us":1649,"replication_time_us":55989}]]}
I20260809 14:30:18.388659 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_cad4e455f76b418f90ffde8a67a9566f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:18.395294 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c955b244a00c442fb7737b4bacb71ed6" } new_table_name: "default.Table_f4339baad1a14705a1f3f35db5081a8b" modify_external_catalogs: true
W20260809 14:30:18.391707 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cad4e455f76b418f90ffde8a67a9566f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.411517 32693 tablet_bootstrap.cc:492] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
W20260809 14:30:18.416318 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b399c4ca4b741098b6ab52bf9cc3e48 consensus state has no leader
I20260809 14:30:18.430110 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ae3dd52b60842a083ae2609d0017b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.431592 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514327321bf142379e8103a4b8c1a937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.433626 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet 239ad14cb3a642e08bace0759bc0b732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:18.435367 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 2727d6a119c248c8aeb7dd25b7a8ccda (DEFAULT_TABLE table=default.Table_0e45100c9aee47e5a5a225d0f0564718 [id=c34ab4b002be4a2c8effce6c44952ebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.469847 32133 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 262) took 1171 ms. Trace:
I20260809 14:30:18.470497 32133 rpcz_store.cc:276] 0809 14:30:17.298205 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.298798 (+ 593us) service_pool.cc:227] Handling call
0809 14:30:17.304619 (+ 5821us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:17.304723 (+ 104us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:18.065875 (+761152us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:18.469055 (+403180us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:18.469778 (+ 723us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:18.448187 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273ca6176a184a719538a9455afded4d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:18.451587 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:18.473897 32331 raft_consensus.cc:2454] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:30:18.475654 32693 ts_tablet_manager.cc:1397] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 1.492s user 0.029s sys 0.032s
I20260809 14:30:18.479220 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc8d0cad33b4778af9fe9247743c38f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:17 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:18.481822 1073 raft_consensus.cc:482] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:18.482446 1073 raft_consensus.cc:504] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:18.483600 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_867a407e542b4a26af79c08ca5d0421f" } modify_external_catalogs: true
I20260809 14:30:18.484158 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_867a407e542b4a26af79c08ca5d0421f" } modify_external_catalogs: true
I20260809 14:30:18.485631 1073 leader_election.cc:288] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:18.488008 32260 leader_election.cc:341] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:18.489051 32260 leader_election.cc:302] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:30:18.489647 32260 raft_consensus.cc:2670] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 14:30:18.500895 32295 consensus_meta.cc:287] T ab045e730f3048a8b63b8962ea191ee7 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.653s user 0.003s sys 0.010s
I20260809 14:30:18.505612 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_46cf63f7388349388957842c51da95d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.507788 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46cf63f7388349388957842c51da95d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.508852 32303 tablet_service.cc:1514] Processing DeleteTablet for tablet e74db9caa7f94059b1bba4a082fe57ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:16 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:18.513125 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_82b4d1f100df47fd8c1c0fed677e4a5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.514667 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82b4d1f100df47fd8c1c0fed677e4a5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.517170 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c5b668a1380d48a893831c7012ce015d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.518674 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5b668a1380d48a893831c7012ce015d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.521790 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad186f0ccf4c15a4f0b16bfffa945c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:18.526062 32295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 260) took 1115 ms. Trace:
I20260809 14:30:18.531313 32705 tablet_bootstrap.cc:492] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:18.531796 32705 ts_tablet_manager.cc:1397] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.807s user 0.017s sys 0.004s
I20260809 14:30:18.537355 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2727d6a119c248c8aeb7dd25b7a8ccda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.532169 32300 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bb4dd74afe4ace94eb1519950a83c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:02 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:18.536365 32304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 258) took 1161 ms. Trace:
W20260809 14:30:18.543211 32260 consensus_peers.cc:487] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:18.545239 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 54941.7us
I20260809 14:30:18.553020 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet bce7aec3225a4fd89b3bfd963b64582a with delete_type TABLET_DATA_DELETED (replaced by tablet 408ee42ea5844741b9ab514cbe6a94e4) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:18.553205 32303 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd with delete_type TABLET_DATA_DELETED (replaced by tablet 23d2b217278c481e8464d00188c70048) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:18.558866 31948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ff4d6273186b4934ae0d6b690122a9fc" } modify_external_catalogs: true
I20260809 14:30:18.559418 31948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ff4d6273186b4934ae0d6b690122a9fc" } modify_external_catalogs: true
W20260809 14:30:18.563622 32260 consensus_peers.cc:487] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:18.567852 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_971214cd6b5048afb1c306f0b0c55219" } new_table_name: "default.Table_a50cf76d58cc49c28888e4e567a71d0d" modify_external_catalogs: true
I20260809 14:30:18.574904 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet ea0a36a6ddc64557b31d0dfef8b5b457
I20260809 14:30:18.587134 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet ea0a36a6ddc64557b31d0dfef8b5b457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:18.589011 32133 tablet_service.cc:1467] Processing CreateTablet for tablet 7b7decba4a9a45f1b3e5bb21c54c924e (DEFAULT_TABLE table=default.Table_971214cd6b5048afb1c306f0b0c55219 [id=261eb132302e4d74bb18d23eea8ee9d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.590312 32133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b7decba4a9a45f1b3e5bb21c54c924e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.591239 1068 tablet.cc:1722] T e28a765e1854486595683c0ac00da24c P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260809 14:30:18.600172 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee6e4c5bf5a464589fcfe3b39220a8b (DEFAULT_TABLE table=default.Table_61526cd7c2b14f0a916e4d916c607a78 [id=c121698e43f84d8eae0939cfec3a36cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.601238 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee6e4c5bf5a464589fcfe3b39220a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.612699 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb90db2df534a3dbca8e5fa643eebb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.620072 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_09716b0be3174d30bc3adf0383d40efd" } new_table_name: "default.Table_f3272d0c348d4d20bd9b5d958bf81581" modify_external_catalogs: true
I20260809 14:30:18.649534 31948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet f74b4deeb32941aeaa3c4b813333c14e
I20260809 14:30:18.552255 32693 raft_consensus.cc:348] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:18.650784 32693 raft_consensus.cc:374] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:18.651029 32693 raft_consensus.cc:729] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:30:18.654573 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b7decba4a9a45f1b3e5bb21c54c924e consensus state has no leader
I20260809 14:30:18.225149 1092 raft_consensus.cc:482] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:18.196590 1082 raft_consensus.cc:504] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:18.669075 1092 raft_consensus.cc:504] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:18.671205 1082 leader_election.cc:288] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:18.297525 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 43931b0436c4472eb12ee1afd46c1900 (DEFAULT_TABLE table=default.Table_e6381d3c953a4958ada0a38267050cc0 [id=03607042808746c19f9e8a26eb7fa135]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.299074 32485 tablet_service.cc:1467] Processing CreateTablet for tablet 514327321bf142379e8103a4b8c1a937 (DEFAULT_TABLE table=default.Table_4eccea50c485475c9e4a6f9e9990cac1 [id=5aef0a1444ad4cba9e08edb834962018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.672636 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43931b0436c4472eb12ee1afd46c1900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.299736 32477 tablet_service.cc:1467] Processing CreateTablet for tablet 0b476fe571374e33a3b4d11057c713ea (DEFAULT_TABLE table=default.Table_8eb42cdceb174196b7743f3d3d12edae [id=9cfa749bc1fa41d89222d651922c7a8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.313380 32481 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc8d0cad33b4778af9fe9247743c38f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:17 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.674945 32481 tablet_service.cc:1514] Processing DeleteTablet for tablet f74b4deeb32941aeaa3c4b813333c14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.676733 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b476fe571374e33a3b4d11057c713ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.679697 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514327321bf142379e8103a4b8c1a937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.331965 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb90db2df534a3dbca8e5fa643eebb7 (DEFAULT_TABLE table=default.Table_5a11d072d3c94cccb6823afc0b64eb83 [id=a0fbc8aa31aa4719a6c5d04575b5c5fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.681269 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb90db2df534a3dbca8e5fa643eebb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.351027 32487 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.352424 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 9bd2360a3974492994a9a5b1ce8d2937 (DEFAULT_TABLE table=default.Table_b341959508524bd796f68f68e26f6c82 [id=7a0e2e1cd3e342de8bad2f981635a828]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.686231 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bd2360a3974492994a9a5b1ce8d2937. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:18.687145 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b802030779d423082b676f99b617a1e consensus state has no leader
I20260809 14:30:18.358628 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8b48c8cde4f11a53eec939a1f8e3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:06 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.380084 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 239ad14cb3a642e08bace0759bc0b732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.382212 32491 tablet_service.cc:1467] Processing CreateTablet for tablet 7ae3dd52b60842a083ae2609d0017b69 (DEFAULT_TABLE table=default.Table_e6685adbd7a7408f9fc60d67696e3b04 [id=33608e9192854140a2022cb0b0aa92f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.403267 32480 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 254) took 1161 ms. Trace:
I20260809 14:30:18.456323 32476 tablet_service.cc:1467] Processing CreateTablet for tablet 2727d6a119c248c8aeb7dd25b7a8ccda (DEFAULT_TABLE table=default.Table_0e45100c9aee47e5a5a225d0f0564718 [id=c34ab4b002be4a2c8effce6c44952ebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.442992 32489 tablet_service.cc:1467] Processing CreateTablet for tablet 8a72259ef0d542e3a9bcb33e3da2cec9 (DEFAULT_TABLE table=default.Table_85987fca61094acbb4a924e9ceeec96a [id=d1d5be11f46c45cb8a1ded80c637e3a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.689826 32491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ae3dd52b60842a083ae2609d0017b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.690479 32489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a72259ef0d542e3a9bcb33e3da2cec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.689235 32480 rpcz_store.cc:276] 0809 14:30:17.233072 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.235455 (+ 2383us) service_pool.cc:227] Handling call
0809 14:30:17.557336 (+321881us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:17.557436 (+ 100us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:18.232605 (+675169us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:18.291792 (+ 59187us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:18.393686 (+101894us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:18.528359 32479 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 259) took 1254 ms. Trace:
I20260809 14:30:18.692602 32479 rpcz_store.cc:276] 0809 14:30:17.273503 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.276070 (+ 2567us) service_pool.cc:227] Handling call
0809 14:30:17.564540 (+288470us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:17.564673 (+ 133us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:18.348029 (+783356us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:18.527457 (+179428us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:18.528263 (+ 806us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:18.541545 32475 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee6e4c5bf5a464589fcfe3b39220a8b (DEFAULT_TABLE table=default.Table_61526cd7c2b14f0a916e4d916c607a78 [id=c121698e43f84d8eae0939cfec3a36cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.545711 32474 tablet_service.cc:1467] Processing CreateTablet for tablet 7b7decba4a9a45f1b3e5bb21c54c924e (DEFAULT_TABLE table=default.Table_971214cd6b5048afb1c306f0b0c55219 [id=261eb132302e4d74bb18d23eea8ee9d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.566597 32691 tablet_bootstrap.cc:492] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:18.585330 32473 tablet_service.cc:1514] Processing DeleteTablet for tablet ea0a36a6ddc64557b31d0dfef8b5b457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.619900 1119 tablet.cc:1722] T e28a765e1854486595683c0ac00da24c P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260809 14:30:18.639583 32470 tablet_service.cc:1467] Processing CreateTablet for tablet a9c969f1ec4744fdb8b810793b099cc8 (DEFAULT_TABLE table=default.Table_7f6aaf330b564e2c8a3126bca9401698 [id=567557fc695846c0aa78b86becb353bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.696228 32691 ts_tablet_manager.cc:1397] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.986s user 0.012s sys 0.015s
I20260809 14:30:18.657017 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:18.643105 32469 tablet_service.cc:1467] Processing CreateTablet for tablet 370a8a83f0c840c1b08d92188e5b62d9 (DEFAULT_TABLE table=default.Table_ac82ee262e4a40bcb9bc60cc74a8861f [id=65aaec21b1e5496a9b094e61bbd4f46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.709070 32475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee6e4c5bf5a464589fcfe3b39220a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.709242 32691 raft_consensus.cc:348] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:18.710969 32691 raft_consensus.cc:374] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:18.711275 32691 raft_consensus.cc:729] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:18.720881 32469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370a8a83f0c840c1b08d92188e5b62d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.722061 32508 raft_consensus.cc:2454] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:18.725288 32476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2727d6a119c248c8aeb7dd25b7a8ccda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.714084 32691 consensus_queue.cc:260] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:18.729984 31944 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet 3d81046d929c48a3a6946f20d8fd7bca (table default.Table_4b848656d0ca4f2b81bd9e56d4540967 [id=f60105c3fd98462b9367f400abca2c15]). Expected version 1 got 0
I20260809 14:30:18.739895 32479 tablet_service.cc:1467] Processing CreateTablet for tablet 401552b9d283466fbfeaf9ad8ed0602f (DEFAULT_TABLE table=default.Table_d7a3df21ac0a4fd2b7cb295edfb62f96 [id=217270d577db418e87628191169460be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.742214 32473 tablet_service.cc:1467] Processing CreateTablet for tablet 48ac456e1a2c48c9851fb139c02ac40b (DEFAULT_TABLE table=default.Table_01106b7a16e142518d374e93aac4da95 [id=f5cc092693d34d67a8a27b147fa767ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.744606 32482 tablet_service.cc:1467] Processing CreateTablet for tablet 35317d7ca48544fda59a4ca273406814 (DEFAULT_TABLE table=default.Table_b62ea93ef63a4b0ba177bdae01cc95b4 [id=9618dc012e8e47c08511b9bf13480892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.746963 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 11f5459874e841e2b39a0e2c5f4d2f87 (DEFAULT_TABLE table=default.Table_d50800cad30d400ba6b4bb95b5f60042 [id=b8301d685310445bbdc4b58b9f3a6e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.740685 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 59134b4266764866aca8030f9240e3e7 (DEFAULT_TABLE table=default.Table_1474c6854a634c82909f0cc97490d2ea [id=9441e797a40845cc8376f490da7f57df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.750144 32470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c969f1ec4744fdb8b810793b099cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.752980 32474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b7decba4a9a45f1b3e5bb21c54c924e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.757910 32487 tablet_service.cc:1467] Processing CreateTablet for tablet a1bfe0fa1bb246ffb434b7aa9652935f (DEFAULT_TABLE table=default.Table_b219214ad9f94dd991137e6a5a35764c [id=7b60c14a5a764cb9abae551857fb55a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.760155 32481 tablet_service.cc:1467] Processing CreateTablet for tablet 408ee42ea5844741b9ab514cbe6a94e4 (DEFAULT_TABLE table=default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.760499 32691 ts_tablet_manager.cc:1428] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.064s user 0.003s sys 0.004s
I20260809 14:30:18.712791 1092 leader_election.cc:288] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:18.774812 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 559954us
I20260809 14:30:18.781512 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_2d7857c7352d430d940bc1294ad4f89a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:18.781912 996 ts_tablet_manager.cc:1918] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:18.782606 996 log.cc:1196] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/9922cad6d10945d5b013a168a3496676
W20260809 14:30:18.783205 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d7857c7352d430d940bc1294ad4f89a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.783636 996 ts_tablet_manager.cc:1939] T 9922cad6d10945d5b013a168a3496676 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:18.785799 31940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_533f32539e9542569c81b5bfe1f0f53e" } modify_external_catalogs: true
I20260809 14:30:18.786368 31940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_533f32539e9542569c81b5bfe1f0f53e" } modify_external_catalogs: true
I20260809 14:30:18.796919 32301 tablet_service.cc:1514] Processing DeleteTablet for tablet c468382a28934d50babf7bbc9a834c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:18.802101 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 142759us
W20260809 14:30:18.803165 32260 consensus_peers.cc:487] T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Couldn't send request to peer 6bed14aff53a41b2bbf5a13ce55dec5a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d592b1c4520467791ffd9d36e0d97fd. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:18.551517 32304 rpcz_store.cc:276] 0809 14:30:17.374810 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.392638 (+ 17828us) service_pool.cc:227] Handling call
0809 14:30:17.394826 (+ 2188us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:17.394953 (+ 127us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:18.202560 (+807607us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:18.535289 (+332729us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:18.536297 (+ 1008us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:18.803758 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_aba0145218af4c9294652147cd9b649a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.805418 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aba0145218af4c9294652147cd9b649a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:18.805976 32443 leader_election.cc:341] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:18.806401 32443 leader_election.cc:302] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:18.807451 1086 raft_consensus.cc:2726] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:18.811195 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401552b9d283466fbfeaf9ad8ed0602f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.818022 32473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ac456e1a2c48c9851fb139c02ac40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.828692 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11f5459874e841e2b39a0e2c5f4d2f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.831162 31938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_b924d95202ef48fb86e02a090323329a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.832620 31938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b924d95202ef48fb86e02a090323329a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.836251 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5a0a3fcdf72041039e8d1c04e4a05537"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:18.837888 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a0a3fcdf72041039e8d1c04e4a05537 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.846838 32693 consensus_queue.cc:260] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:18.854256 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_6cb89cdda08047ada14df086bf662194" } modify_external_catalogs: true
I20260809 14:30:18.854754 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_6cb89cdda08047ada14df086bf662194" } modify_external_catalogs: true
W20260809 14:30:18.853281 32439 leader_election.cc:341] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:18.859377 32443 leader_election.cc:302] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:18.865461 32139 tablet_service.cc:1467] Processing CreateTablet for tablet a9c969f1ec4744fdb8b810793b099cc8 (DEFAULT_TABLE table=default.Table_7f6aaf330b564e2c8a3126bca9401698 [id=567557fc695846c0aa78b86becb353bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.876753 1086 raft_consensus.cc:2781] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:18.877157 1086 raft_consensus.cc:482] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:18.877563 31945 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_d841d100269948ca8f7fc467c85a3b0d" } new_table_name: "default.Table_cad12e9e9662472d80abf06e813261b3" modify_external_catalogs: true
I20260809 14:30:18.877506 1086 raft_consensus.cc:3037] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:18.879446 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59134b4266764866aca8030f9240e3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.879326 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 370a8a83f0c840c1b08d92188e5b62d9 (DEFAULT_TABLE table=default.Table_ac82ee262e4a40bcb9bc60cc74a8861f [id=65aaec21b1e5496a9b094e61bbd4f46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.882596 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35317d7ca48544fda59a4ca273406814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.882781 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c969f1ec4744fdb8b810793b099cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.884786 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1bfe0fa1bb246ffb434b7aa9652935f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.885386 32485 tablet_service.cc:1467] Processing CreateTablet for tablet 5b399c4ca4b741098b6ab52bf9cc3e48 (DEFAULT_TABLE table=default.Table_f4339baad1a14705a1f3f35db5081a8b [id=6c55b37d949a4648bd2d31852bb7727d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.886307 31940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 04b74ba6259e47569d13dfe62d14a4af
I20260809 14:30:18.886905 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370a8a83f0c840c1b08d92188e5b62d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.887287 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408ee42ea5844741b9ab514cbe6a94e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.899900 32691 tablet_bootstrap.cc:492] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:18.901233 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 04b74ba6259e47569d13dfe62d14a4af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:18.907116 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b399c4ca4b741098b6ab52bf9cc3e48. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:18.913282 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 37a3a71874fd44bf865b11c05e84ca84 (table default.Table_60947825f97f4a7981e3301069cbd4ca [id=2cf0c0e1fb6f40c29bb391b38988e91a], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 37a3a71874fd44bf865b11c05e84ca84 consensus state has no leader
I20260809 14:30:18.913802 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_45d346a083d04fc6b8a9d8631c65bff1" } new_table_name: "default.Table_ed7d64f6d7b644c48b0190ff65c56551" modify_external_catalogs: true
I20260809 14:30:18.918594 32140 tablet_service.cc:1514] Processing DeleteTablet for tablet f74b4deeb32941aeaa3c4b813333c14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:18.944499 1010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:30:18.947986 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet c4f4c1eb916c4b51a5e176d541da8574
I20260809 14:30:18.945849 1010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
W20260809 14:30:18.953284 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 93995.8us
I20260809 14:30:18.956710 32296 tablet_service.cc:1467] Processing CreateTablet for tablet 8a72259ef0d542e3a9bcb33e3da2cec9 (DEFAULT_TABLE table=default.Table_85987fca61094acbb4a924e9ceeec96a [id=d1d5be11f46c45cb8a1ded80c637e3a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.958149 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a72259ef0d542e3a9bcb33e3da2cec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.960767 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet c4f4c1eb916c4b51a5e176d541da8574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:44626
W20260809 14:30:18.969691 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23d2b217278c481e8464d00188c70048 consensus state has no leader
I20260809 14:30:18.972213 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 48ac456e1a2c48c9851fb139c02ac40b (DEFAULT_TABLE table=default.Table_01106b7a16e142518d374e93aac4da95 [id=f5cc092693d34d67a8a27b147fa767ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.973302 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ac456e1a2c48c9851fb139c02ac40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.976537 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad186f0ccf4c15a4f0b16bfffa945c" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:18.978397 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8b48c8cde4f11a53eec939a1f8e3d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:18.962738 1120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:30:18.987031 32304 tablet_service.cc:1467] Processing CreateTablet for tablet 0b476fe571374e33a3b4d11057c713ea (DEFAULT_TABLE table=default.Table_8eb42cdceb174196b7743f3d3d12edae [id=9cfa749bc1fa41d89222d651922c7a8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.988044 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 401552b9d283466fbfeaf9ad8ed0602f (DEFAULT_TABLE table=default.Table_d7a3df21ac0a4fd2b7cb295edfb62f96 [id=217270d577db418e87628191169460be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.984651 1120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:30:18.997234 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 43931b0436c4472eb12ee1afd46c1900 (DEFAULT_TABLE table=default.Table_e6381d3c953a4958ada0a38267050cc0 [id=03607042808746c19f9e8a26eb7fa135]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:18.998315 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43931b0436c4472eb12ee1afd46c1900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.998214 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 59134b4266764866aca8030f9240e3e7 (DEFAULT_TABLE table=default.Table_1474c6854a634c82909f0cc97490d2ea [id=9441e797a40845cc8376f490da7f57df]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:18.846396 1009 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s user 0.009s sys 0.014s
I20260809 14:30:19.000923 32303 tablet_service.cc:1467] Processing CreateTablet for tablet 514327321bf142379e8103a4b8c1a937 (DEFAULT_TABLE table=default.Table_4eccea50c485475c9e4a6f9e9990cac1 [id=5aef0a1444ad4cba9e08edb834962018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.001931 32303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514327321bf142379e8103a4b8c1a937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.013379 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b476fe571374e33a3b4d11057c713ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.021589 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59134b4266764866aca8030f9240e3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.022434 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401552b9d283466fbfeaf9ad8ed0602f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:18.972065 32162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d81046d929c48a3a6946f20d8fd7bca" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:19.034197 32443 leader_election.cc:341] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:19.027865 32705 raft_consensus.cc:348] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.036759 32705 raft_consensus.cc:374] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:19.037029 32705 raft_consensus.cc:729] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:19.040242 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:19.040669 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 11f5459874e841e2b39a0e2c5f4d2f87 (DEFAULT_TABLE table=default.Table_d50800cad30d400ba6b4bb95b5f60042 [id=b8301d685310445bbdc4b58b9f3a6e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.041698 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11f5459874e841e2b39a0e2c5f4d2f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.042316 32705 consensus_queue.cc:260] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.045169 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 35317d7ca48544fda59a4ca273406814 (DEFAULT_TABLE table=default.Table_b62ea93ef63a4b0ba177bdae01cc95b4 [id=9618dc012e8e47c08511b9bf13480892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.056241 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35317d7ca48544fda59a4ca273406814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.062453 32705 ts_tablet_manager.cc:1428] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.530s user 0.009s sys 0.001s
W20260809 14:30:19.054975 32163 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s user 0.011s sys 0.026s
W20260809 14:30:19.069969 32163 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.812s user 0.011s sys 0.026s
W20260809 14:30:19.062781 32705 ts_tablet_manager.cc:1460] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Tablet startup took 530ms
I20260809 14:30:19.071460 1068 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:41544 (request call id 258) took 1816 ms. Trace:
W20260809 14:30:19.070710 32705 ts_tablet_manager.cc:1462] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Trace:
0809 14:30:01.769272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.770266 (+ 994us) service_pool.cc:227] Handling call
0809 14:30:01.786169 (+ 15903us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:01.786258 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:02.059487 (+273229us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.133142 (+ 73655us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.133599 (+ 457us) inbound_call.cc:173] Queueing success response
0809 14:30:17.501146 (+15367547us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 14:30:18.532076 (+1030930us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 14:30:19.025990 (+493914us) tablet_replica.cc:259] Starting instrumentation
0809 14:30:19.026568 (+ 578us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":6972,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":15268,"tablet-open.queue_time_us":15367571}
I20260809 14:30:19.072703 32705 tablet_bootstrap.cc:492] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:19.077333 32132 consensus_meta.cc:287] T 0b476fe571374e33a3b4d11057c713ea P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.635s user 0.004s sys 0.008s
W20260809 14:30:19.090370 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 285902us
I20260809 14:30:19.093480 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9e117bba409147cf8e8ec3e5443a321c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:19.096546 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_657bee986ba04a85aa02aff63c665910" } modify_external_catalogs: true
I20260809 14:30:19.097064 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_657bee986ba04a85aa02aff63c665910" } modify_external_catalogs: true
I20260809 14:30:19.100085 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9f890199a5164be68c5b41ad6bc0ddb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.101492 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f890199a5164be68c5b41ad6bc0ddb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:18.551644 32295 rpcz_store.cc:276] 0809 14:30:17.411201 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.411446 (+ 245us) service_pool.cc:227] Handling call
0809 14:30:17.414414 (+ 2968us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:17.414611 (+ 197us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:17.847033 (+432422us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:18.512275 (+665242us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:18.525984 (+ 13709us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:19.106137 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_45f0fdfd0b17414694813aa5025f9759"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:19.107890 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_189adb7a0cf1426aaff4dd77f5991e8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:19.110339 967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee93a411a1b04d38998018f8c6be59e0. Latest consensus state: current_term: 6 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:30:19.111233 967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 14:30:18.999656 1009 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s user 0.009s sys 0.014s
W20260809 14:30:19.119375 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 127716us
I20260809 14:30:19.120801 32300 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb90db2df534a3dbca8e5fa643eebb7 (DEFAULT_TABLE table=default.Table_5a11d072d3c94cccb6823afc0b64eb83 [id=a0fbc8aa31aa4719a6c5d04575b5c5fc]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:19.128516 32299 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.650s user 0.000s sys 0.018s
W20260809 14:30:19.128950 32299 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.650s user 0.000s sys 0.018s
W20260809 14:30:19.129113 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_189adb7a0cf1426aaff4dd77f5991e8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:19.132495 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45f0fdfd0b17414694813aa5025f9759 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.140340 32141 tablet_service.cc:1467] Processing CreateTablet for tablet 408ee42ea5844741b9ab514cbe6a94e4 (DEFAULT_TABLE table=default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.141012 32134 tablet_service.cc:1467] Processing CreateTablet for tablet a1bfe0fa1bb246ffb434b7aa9652935f (DEFAULT_TABLE table=default.Table_b219214ad9f94dd991137e6a5a35764c [id=7b60c14a5a764cb9abae551857fb55a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.141398 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408ee42ea5844741b9ab514cbe6a94e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.141992 32134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1bfe0fa1bb246ffb434b7aa9652935f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.148360 32163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.30.252.66:52479 (request call id 172) took 2047 ms. Trace:
I20260809 14:30:19.148743 32163 rpcz_store.cc:276] 0809 14:30:17.100967 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.107325 (+ 6358us) service_pool.cc:227] Handling call
0809 14:30:17.107799 (+ 474us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 14:30:17.960556 (+852757us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 14:30:17.972530 (+ 11974us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 14:30:18.056853 (+ 84323us) log.cc:842] Serialized 26 byte log entry
0809 14:30:18.257536 (+200683us) raft_consensus.cc:1631] Marking committed up to 1
0809 14:30:19.070605 (+813069us) log.cc:842] Serialized 14 byte log entry
0809 14:30:19.148031 (+ 77426us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 14:30:19.148126 (+ 95us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 14:30:19.148168 (+ 42us) raft_consensus.cc:1671] finished
0809 14:30:19.148183 (+ 15us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 14:30:19.148299 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":8328,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":418847,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":232,"thread_start_us":1251608,"threads_started":3,"wal-append.queue_time_us":195270}
W20260809 14:30:19.154004 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e117bba409147cf8e8ec3e5443a321c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.158541 31939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_cc226a9bef8a4d6bb50d7bf50e57193c" } modify_external_catalogs: true
I20260809 14:30:19.159101 31939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_cc226a9bef8a4d6bb50d7bf50e57193c" } modify_external_catalogs: true
I20260809 14:30:19.163609 31943 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_cdb56a7b91ec4bf2ad8f560243ecb761" } new_table_name: "default.Table_558c3b629f83480ea933b6c8d7ee9d1d" modify_external_catalogs: true
I20260809 14:30:19.151508 1014 raft_consensus.cc:482] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:19.164969 1014 raft_consensus.cc:504] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.173048 31946 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_93526ced6a324be680b48ad0d4587fe6" } new_table_name: "default.Table_35322bfb7f9a4e28a3ca0725dedb6cf3" modify_external_catalogs: true
I20260809 14:30:19.071775 1068 rpcz_store.cc:276] 0809 14:30:17.255537 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.265276 (+ 9739us) service_pool.cc:227] Handling call
0809 14:30:19.071375 (+1806099us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:17.849537 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:17.850333 (+ 796us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:17.850403 (+ 70us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:17.850705 (+ 302us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:17.850885 (+ 180us) alter_schema_op.cc:126] START. Timestamp: 1786285817850166
0809 14:30:17.850933 (+ 48us) op_driver.cc:331] REPLICATION: starting
0809 14:30:17.851880 (+ 947us) log.cc:842] Serialized 173 byte log entry
0809 14:30:18.275819 (+423939us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400142ba0. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181af0 00007fb6c718c028 00007fb6c718bc6a 00007fb6c71f7cd5 00007fb6c71f8179 00007fb6c7659ade 00007fb6c76592b8 00007fb6c765dba1 00007fb6c765db59 00007fb6c765dae9 00007fb6c765dab1
0809 14:30:18.590797 (+314978us) op_driver.cc:444] REPLICATION: finished
0809 14:30:18.591159 (+ 362us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:18.592400 (+ 1241us) log.cc:842] Serialized 5 byte log entry
0809 14:30:19.014248 (+421848us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.071133 (+ 56885us) log.cc:842] Serialized 14 byte log entry
0809 14:30:19.071230 (+ 97us) alter_schema_op.cc:71] Released schema lock
0809 14:30:19.071254 (+ 24us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":194,"mutex_wait_us":24587,"prepare.queue_time_us":536510,"prepare.run_cpu_time_us":16520,"prepare.run_wall_time_us":490930,"raft.queue_time_us":59087,"raft.run_cpu_time_us":25,"raft.run_wall_time_us":27,"replication_time_us":739748,"spinlock_wait_cycles":287714560,"thread_start_us":576930,"threads_started":2,"wal-append.queue_time_us":267700}]]}
I20260809 14:30:19.188021 32148 tablet_service.cc:1514] Processing DeleteTablet for tablet c4f4c1eb916c4b51a5e176d541da8574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:19.189226 1014 leader_election.cc:288] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:19.199287 31943 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8ecf7edec2b48dcbaea4f1413b44f95 has no consensus state
I20260809 14:30:19.212913 32132 tablet_service.cc:1514] Processing DeleteTablet for tablet 04b74ba6259e47569d13dfe62d14a4af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:19.216451 31946 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6812d380b8d143a0bc35c4dc9b2aaeeb has no consensus state
I20260809 14:30:19.234102 32145 tablet_service.cc:1467] Processing CreateTablet for tablet 5b399c4ca4b741098b6ab52bf9cc3e48 (DEFAULT_TABLE table=default.Table_f4339baad1a14705a1f3f35db5081a8b [id=6c55b37d949a4648bd2d31852bb7727d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.236819 32693 ts_tablet_manager.cc:1428] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.755s user 0.008s sys 0.000s
W20260809 14:30:19.237216 32693 ts_tablet_manager.cc:1460] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b: Tablet startup took 755ms
I20260809 14:30:19.245249 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 9bd2360a3974492994a9a5b1ce8d2937 (DEFAULT_TABLE table=default.Table_b341959508524bd796f68f68e26f6c82 [id=7a0e2e1cd3e342de8bad2f981635a828]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:19.255302 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 164100us
I20260809 14:30:19.261415 1088 consensus_queue.cc:1035] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 14:30:19.267940 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb90db2df534a3dbca8e5fa643eebb7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:19.271503 1001 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.523s user 0.005s sys 0.008s
W20260809 14:30:19.272282 1001 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.523s user 0.005s sys 0.008s
I20260809 14:30:19.289183 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8b48c8cde4f11a53eec939a1f8e3d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:19.292021 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d81046d929c48a3a6946f20d8fd7bca" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:19.294072 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bd2360a3974492994a9a5b1ce8d2937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.313627 32489 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 276) took 1090 ms. Trace:
W20260809 14:30:19.314750 32443 leader_election.cc:341] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:19.316079 32443 leader_election.cc:341] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:19.241894 32693 ts_tablet_manager.cc:1462] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b: Trace:
0809 14:30:03.775216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.775904 (+ 688us) service_pool.cc:227] Handling call
0809 14:30:03.777468 (+ 1564us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:03.777558 (+ 90us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:03.799395 (+ 21837us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.825548 (+ 26153us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.866173 (+ 40625us) inbound_call.cc:173] Queueing success response
0809 14:30:16.983488 (+13117315us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 14:30:18.481258 (+1497770us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 14:30:18.519515 (+ 38257us) tablet_replica.cc:259] Starting instrumentation
0809 14:30:18.520100 (+ 585us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":133940,"raft.queue_time_us":250747,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":137774,"spinlock_wait_cycles":460160,"tablet-open.queue_time_us":13117381}
I20260809 14:30:19.314057 32489 rpcz_store.cc:276] 0809 14:30:18.223702 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.442210 (+218508us) service_pool.cc:227] Handling call
0809 14:30:18.690131 (+247921us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.690198 (+ 67us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:18.731739 (+ 41541us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.136314 (+404575us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.313552 (+177238us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:19.337164 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 7ae3dd52b60842a083ae2609d0017b69 (DEFAULT_TABLE table=default.Table_e6685adbd7a7408f9fc60d67696e3b04 [id=33608e9192854140a2022cb0b0aa92f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.338346 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ae3dd52b60842a083ae2609d0017b69. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:19.342409 32478 consensus_meta.cc:287] T 782ba14ca8bc41429a086d49cc40434f P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 1.079s user 0.014s sys 0.007s
I20260809 14:30:19.345327 32693 tablet_bootstrap.cc:492] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
W20260809 14:30:19.353498 32491 consensus_meta.cc:287] T 7ae3dd52b60842a083ae2609d0017b69 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.555s user 0.007s sys 0.004s
I20260809 14:30:19.353902 32295 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:19.357803 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 251) took 3209 ms. Trace:
I20260809 14:30:19.396392 32491 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 284) took 1015 ms. Trace:
I20260809 14:30:19.395480 996 rpcz_store.cc:276] 0809 14:30:16.148931 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:16.149049 (+ 118us) service_pool.cc:227] Handling call
0809 14:30:17.962852 (+1813803us) ts_tablet_manager.cc:1145] Deleting tablet 9922cad6d10945d5b013a168a3496676
0809 14:30:17.962916 (+ 64us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:18.572567 (+609651us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:18.781518 (+208951us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.357732 (+576214us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10876,"tablet-delete.queue_time_us":1805085}
I20260809 14:30:19.396847 32491 rpcz_store.cc:276] 0809 14:30:18.380726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.380943 (+ 217us) service_pool.cc:227] Handling call
0809 14:30:18.689325 (+308382us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.689389 (+ 64us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:18.796537 (+107148us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.395377 (+598840us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.395889 (+ 512us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:19.398890 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 252) took 3224 ms. Trace:
I20260809 14:30:19.399351 1020 ts_tablet_manager.cc:1918] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:19.399297 996 rpcz_store.cc:276] 0809 14:30:16.174051 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:16.174251 (+ 200us) service_pool.cc:227] Handling call
0809 14:30:19.398609 (+3224358us) ts_tablet_manager.cc:1145] Deleting tablet bce7aec3225a4fd89b3bfd963b64582a
0809 14:30:19.398660 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:19.398836 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3223524}
I20260809 14:30:19.404083 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_dad01c31cb0242f9a295c92854814711" } new_table_name: "default.Table_a7bb2c0d2c824a9ba439373e1ae5525d" modify_external_catalogs: true
I20260809 14:30:19.413340 996 tablet_replica.cc:330] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:19.414860 996 raft_consensus.cc:2229] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:19.415838 996 raft_consensus.cc:2258] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:19.417938 996 ts_tablet_manager.cc:1905] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:19.422108 31940 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b86f8f5a1acc4bacb6b93b4acd413b0f" } new_table_name: "default.Table_423d2129e8fa4a59a8b4a7e853467ffa" modify_external_catalogs: true
I20260809 14:30:19.427143 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_0e4ff3df5ea743e4b5c4db07bdd03d1f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.435510 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 295759us
I20260809 14:30:19.437024 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_424b18fbca56430bb3e23219c6153e97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.439138 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_424b18fbca56430bb3e23219c6153e97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:19.442802 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e4ff3df5ea743e4b5c4db07bdd03d1f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.447160 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b399c4ca4b741098b6ab52bf9cc3e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.446883 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 9922cad6d10945d5b013a168a3496676 (table default.Table_8792541442324807881089a25803c23e [id=11d64f84c8854864b5ea0f591af8d79b]) successfully deleted
W20260809 14:30:19.449309 31930 catalog_manager.cc:4808] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet bce7aec3225a4fd89b3bfd963b64582a because the tablet was not found. No further retry: Not found: Tablet not found: bce7aec3225a4fd89b3bfd963b64582a
W20260809 14:30:19.458770 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 782ba14ca8bc41429a086d49cc40434f consensus state has no leader
I20260809 14:30:19.381625 32478 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 258) took 2108 ms. Trace:
W20260809 14:30:19.359596 32483 consensus_meta.cc:287] T 3fb90db2df534a3dbca8e5fa643eebb7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.604s user 0.006s sys 0.002s
I20260809 14:30:19.462513 32483 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 280) took 1131 ms. Trace:
I20260809 14:30:19.462798 32483 rpcz_store.cc:276] 0809 14:30:18.331147 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.331271 (+ 124us) service_pool.cc:227] Handling call
0809 14:30:18.680897 (+349626us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.680975 (+ 78us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:18.755490 (+ 74515us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.462087 (+706597us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.462459 (+ 372us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:19.463461 32705 tablet_bootstrap.cc:654] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:19.461340 32478 rpcz_store.cc:276] 0809 14:30:17.273204 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.273716 (+ 512us) service_pool.cc:227] Handling call
0809 14:30:17.560514 (+286798us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:17.560643 (+ 129us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:18.262882 (+702239us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.343159 (+1080277us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.381532 (+ 38373us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:19.471818 1004 ts_tablet_manager.cc:1918] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:19.472586 1004 log.cc:1196] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/29fdaef8a95e4aa0b23018d384442d16
I20260809 14:30:19.475069 32126 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:41528 (request call id 54) took 1763 ms. Trace:
I20260809 14:30:19.475627 1004 ts_tablet_manager.cc:1939] T 29fdaef8a95e4aa0b23018d384442d16 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:19.481266 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 276) took 1630 ms. Trace:
I20260809 14:30:19.386831 1010 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
W20260809 14:30:19.492645 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 372626us
W20260809 14:30:19.493631 32439 leader_election.cc:341] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:19.494164 32439 leader_election.cc:302] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:19.501619 31940 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 32cb7842617a4779bf110e4ea254e011 consensus state has no leader
I20260809 14:30:19.502337 1137 raft_consensus.cc:482] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:19.508034 1082 raft_consensus.cc:2726] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:19.514267 1004 rpcz_store.cc:276] 0809 14:30:17.851012 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.853858 (+ 2846us) service_pool.cc:227] Handling call
0809 14:30:18.175792 (+321934us) ts_tablet_manager.cc:1145] Deleting tablet 29fdaef8a95e4aa0b23018d384442d16
0809 14:30:18.175833 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:18.918027 (+742194us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.471541 (+553514us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.481174 (+ 9633us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":321267}
I20260809 14:30:19.516428 1004 tablet_replica.cc:330] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:19.524317 1020 log.cc:1196] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/29fdaef8a95e4aa0b23018d384442d16
I20260809 14:30:19.525424 1020 ts_tablet_manager.cc:1939] T 29fdaef8a95e4aa0b23018d384442d16 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:19.526508 1020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 244) took 3539 ms. Trace:
W20260809 14:30:19.393667 1117 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s user 0.003s sys 0.023s
W20260809 14:30:19.527230 1117 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s user 0.003s sys 0.024s
W20260809 14:30:19.540843 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 215183us
I20260809 14:30:19.550622 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 29fdaef8a95e4aa0b23018d384442d16 (table default.Table_3c5c49afb5814baf8e9f3f33f5beac33 [id=0c17902222b34f16aab8173dbd666e86]) successfully deleted
I20260809 14:30:19.566918 32693 tablet_bootstrap.cc:654] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:19.502946 1137 raft_consensus.cc:504] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.582329 31940 catalog_manager.cc:5587] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
I20260809 14:30:19.585593 32477 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 277) took 1289 ms. Trace:
I20260809 14:30:19.586378 32477 rpcz_store.cc:276] 0809 14:30:18.296080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.298688 (+ 2608us) service_pool.cc:227] Handling call
0809 14:30:18.676076 (+377388us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.676195 (+ 119us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.346694 (+670499us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.584527 (+237833us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.585469 (+ 942us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22342}
I20260809 14:30:19.596496 32691 tablet_bootstrap.cc:654] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:19.599138 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 105546us
W20260809 14:30:19.600498 32439 leader_election.cc:341] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:19.601593 32439 leader_election.cc:302] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:19.603099 1082 raft_consensus.cc:2726] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:19.610504 32439 leader_election.cc:341] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:19.612505 32439 leader_election.cc:302] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:19.598666 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_4f45e7ed5834412c9ffa9f4be4afc08a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.632710 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f45e7ed5834412c9ffa9f4be4afc08a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.613164 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_44d99dd47a7a4e5ca147890da0688db7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.634753 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44d99dd47a7a4e5ca147890da0688db7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.640175 1092 raft_consensus.cc:2726] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:19.641014 1092 raft_consensus.cc:482] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:19.641745 1092 raft_consensus.cc:504] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.504738 32126 rpcz_store.cc:276] 0809 14:30:17.710655 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.710800 (+ 145us) service_pool.cc:227] Handling call
0809 14:30:18.277888 (+567088us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54000a3a54. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181471 00007fb6c71f6268 00007fb6c81e26cd 00007fb6c6a8ae04 00007fb6c6a8ad92 00007fb6c6a8acc1 00007fb6c6a8ac3c 00007fb6c6a89ef2 00007fb6c4fab94c 00007fb6c4faad86 00007fb6c4faa72f
0809 14:30:19.153768 (+875880us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54001220d4. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181471 00007fb6c71f6268 00007fb6c81e26cd 00007fb6c6a8ae04 00007fb6c6a8ad92 00007fb6c6a8acc1 00007fb6c6a8ac3c 00007fb6c6a89ef2 00007fb6c4fab94c 00007fb6c4faad86 00007fb6c4faa72f
0809 14:30:19.474034 (+320266us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":940784768}
I20260809 14:30:19.645146 1140 raft_consensus.cc:482] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:19.646040 1092 leader_election.cc:288] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:19.645927 1140 raft_consensus.cc:504] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.651553 1020 rpcz_store.cc:276] 0809 14:30:15.986714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.987611 (+ 897us) service_pool.cc:227] Handling call
0809 14:30:17.780182 (+1792571us) ts_tablet_manager.cc:1145] Deleting tablet 29fdaef8a95e4aa0b23018d384442d16
0809 14:30:17.780233 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:18.421516 (+641283us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.399011 (+977495us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.526450 (+127439us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1792062}
I20260809 14:30:19.653028 1020 tablet_replica.cc:330] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:19.662132 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_60f882b13b3d48f28134afe6dfed9f10" } new_table_name: "default.Table_eaa055a1bfde40768487fe94df03d134" modify_external_catalogs: true
I20260809 14:30:19.652176 32469 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 290) took 1011 ms. Trace:
W20260809 14:30:19.671810 1086 consensus_meta.cc:287] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.794s user 0.003s sys 0.023s
I20260809 14:30:19.669543 32469 rpcz_store.cc:276] 0809 14:30:18.640959 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.641218 (+ 259us) service_pool.cc:227] Handling call
0809 14:30:18.710201 (+ 68983us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.710308 (+ 107us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.342153 (+631845us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.574886 (+232733us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.647534 (+ 72648us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:19.706416 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_3344e8d1d6c347aaa907473e456adb3b" } new_table_name: "default.Table_23ccc6f75d894549beac50decbabd886" modify_external_catalogs: true
I20260809 14:30:19.560267 32135 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 283) took 1302 ms. Trace:
I20260809 14:30:19.714814 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 060e3ed3be3140559a59f7204ce0a2b6 (DEFAULT_TABLE table=default.Table_089b0a96a71948bd8bc0df4624d303f2 [id=3016ad78c9fd4f8f89874c7da5dcf3c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.715924 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e3ed3be3140559a59f7204ce0a2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.720825 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 7 candidate_status { last_received { term: 6 index: 320 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:30:19.694366 32133 tablet_service.cc:1467] Processing CreateTablet for tablet e76a109e93dd47b18e4f437b3965c44c (DEFAULT_TABLE table=default.Table_46cf63f7388349388957842c51da95d6 [id=33a64d0e281e4939bef7f46be25aef43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.726408 32133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e76a109e93dd47b18e4f437b3965c44c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:19.440398 32095 leader_election.cc:341] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 2f171260cb764ac284aef85e4219767e
I20260809 14:30:19.464864 1129 raft_consensus.cc:482] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 14:30:19.739444 32095 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 299058us
I20260809 14:30:19.739500 1129 raft_consensus.cc:504] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.741726 31940 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_20cad555a34f4f70a19814ffcb820bbe [id=20d2073f354649b3b4bc4d583660c93a] alter complete (version 4)
I20260809 14:30:19.743777 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac7afd531a746cab7618ead98b4c4d9" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:19.714839 32135 rpcz_store.cc:276] 0809 14:30:18.258092 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.321410 (+ 63318us) service_pool.cc:227] Handling call
0809 14:30:18.456798 (+135388us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.459667 (+ 2869us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.466329 (+1006662us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.559783 (+ 93454us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.560207 (+ 424us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:19.745095 32162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdfc56edcd34696a7b806b0ee34d594" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:19.746894 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce9848ee8f7d45cba615b06b9c826afb" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:19.518539 1004 raft_consensus.cc:2229] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:19.748646 1004 raft_consensus.cc:2258] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:19.691815 32705 tablet_bootstrap.cc:492] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:19.750182 32705 ts_tablet_manager.cc:1397] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.678s user 0.010s sys 0.007s
I20260809 14:30:19.703327 32146 tablet_service.cc:1467] Processing CreateTablet for tablet d2da78cb058d400495986a23be4b5d56 (DEFAULT_TABLE table=default.Table_6d21f9a0c69445bfb2882750ba0792dc [id=ac6ecfcb8fe04daaa87fdf49567041af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.753844 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2da78cb058d400495986a23be4b5d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.754839 1137 leader_election.cc:288] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:19.752559 1020 raft_consensus.cc:2229] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:19.755915 1020 raft_consensus.cc:2258] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:19.757299 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c564d1dbe35c421dbd0d9ee2d9b2ece5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.759347 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c564d1dbe35c421dbd0d9ee2d9b2ece5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.759372 32480 tablet_service.cc:1467] Processing CreateTablet for tablet d2da78cb058d400495986a23be4b5d56 (DEFAULT_TABLE table=default.Table_6d21f9a0c69445bfb2882750ba0792dc [id=ac6ecfcb8fe04daaa87fdf49567041af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.760567 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2da78cb058d400495986a23be4b5d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.783272 31946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_65ed99b110124e74b05aaccdc8ecd9b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.785264 31946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65ed99b110124e74b05aaccdc8ecd9b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.797410 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 48ad4aee960b4ca5bebeab8ba4899c29 (DEFAULT_TABLE table=default.Table_3344e8d1d6c347aaa907473e456adb3b [id=02a21b34d14e429d954e11d78733643f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.798985 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ad4aee960b4ca5bebeab8ba4899c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.807984 1014 raft_consensus.cc:482] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:19.808763 1014 raft_consensus.cc:504] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.812755 1014 leader_election.cc:288] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:19.813243 1129 leader_election.cc:288] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:19.806461 1004 ts_tablet_manager.cc:1905] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:19.818095 32439 leader_election.cc:341] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:19.825529 32705 raft_consensus.cc:348] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.831722 31940 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet d9f01d25eac34d4d998dc457be6a5d9b (table default.Table_0dcb148ef7df472db8e57a80264a25c2 [id=f2f1c821dd6a4e1d9ebf893bc161805c]). Expected version 1 got 0
W20260809 14:30:19.834894 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c423cc085ce9488697bb63159228541f consensus state has no leader
I20260809 14:30:19.835204 31939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 666bea53f14541368730f69402c2b411
I20260809 14:30:19.836498 32705 raft_consensus.cc:374] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:19.836835 32705 raft_consensus.cc:729] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:19.828742 32145 tablet_service.cc:1467] Processing CreateTablet for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab (DEFAULT_TABLE table=default.Table_8405c747a7ce4e579d0fe4c17912d70b [id=896010e13dec44f58076ec15ecdbc58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.838517 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.672394 1086 raft_consensus.cc:504] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.853040 31945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 90e67f393f7a4bb987edc99aa86a9c90
W20260809 14:30:19.845702 32439 leader_election.cc:341] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:19.855935 32439 leader_election.cc:341] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:19.846297 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet 666bea53f14541368730f69402c2b411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:19 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:19.847781 32477 tablet_service.cc:1467] Processing CreateTablet for tablet 48ad4aee960b4ca5bebeab8ba4899c29 (DEFAULT_TABLE table=default.Table_3344e8d1d6c347aaa907473e456adb3b [id=02a21b34d14e429d954e11d78733643f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.849174 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:19.862599 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 3d7cbb2bb26c4bd7a089648007d7f041 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:19.881896 32569 heartbeater.cc:503] Master 127.30.252.126:43223 was elected leader, sending a full tablet report...
I20260809 14:30:19.890568 32473 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 292) took 1162 ms. Trace:
I20260809 14:30:19.777405 1140 leader_election.cc:288] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:19.905258 996 ts_tablet_manager.cc:1918] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:19.905728 996 log.cc:1196] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/94ff6ff27f6144d08d68a5e79c12963f
I20260809 14:30:19.906750 996 ts_tablet_manager.cc:1939] T 94ff6ff27f6144d08d68a5e79c12963f P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:19.891214 32473 rpcz_store.cc:276] 0809 14:30:18.727953 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.732709 (+ 4756us) service_pool.cc:227] Handling call
0809 14:30:18.816541 (+ 83832us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.816751 (+ 210us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.441451 (+624700us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.670700 (+229249us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.890448 (+219748us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":39381}
W20260809 14:30:19.914665 1107 log.cc:924] Time spent T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: Append to log took a long time: real 1.347s user 0.025s sys 0.018s
W20260809 14:30:19.917946 31931 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5b802030779d423082b676f99b617a1e (table default.Table_f3272d0c348d4d20bd9b5d958bf81581 [id=4ec3081646e84f14ba0605bbb51e2f11], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5b802030779d423082b676f99b617a1e consensus state has no leader
I20260809 14:30:19.909498 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ad4aee960b4ca5bebeab8ba4899c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.910108 32481 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 298) took 1151 ms. Trace:
I20260809 14:30:19.912288 32469 tablet_service.cc:1467] Processing CreateTablet for tablet e76a109e93dd47b18e4f437b3965c44c (DEFAULT_TABLE table=default.Table_46cf63f7388349388957842c51da95d6 [id=33a64d0e281e4939bef7f46be25aef43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:19.923156 1086 leader_election.cc:288] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:19.926275 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet 90e67f393f7a4bb987edc99aa86a9c90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:19 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:19.936694 32469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e76a109e93dd47b18e4f437b3965c44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:19.927603 32481 rpcz_store.cc:276] 0809 14:30:18.759356 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.759473 (+ 117us) service_pool.cc:227] Handling call
0809 14:30:18.886794 (+127321us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.886901 (+ 107us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.515424 (+628523us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.850037 (+334613us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.900724 (+ 50687us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b5800001898. stack: 00007f555ab48033 00007f555ab47f30 00007f555a6c9534 00000000004db2b4 00000000004db249 00007f555a9bef11 00007f555ab42851 00007f555ab42e82 00007f5563a028b9 00007f5562a365d8 00007f5562afa063 00007f5562af5e2f 00007f5562af65ed 00007f5562aa7869 00007f5561199c44 00007f5561199bd2
0809 14:30:19.910051 (+ 9327us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16325,"spinlock_wait_cycles":9715584}
I20260809 14:30:19.941947 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_d49b8d515bb247e2b2114653b0d60633"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.943543 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d49b8d515bb247e2b2114653b0d60633 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.941989 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_cf369e8c18034e71a7603e77b8fd94ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:19.947245 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf369e8c18034e71a7603e77b8fd94ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:19.952206 32476 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 285) took 1498 ms. Trace:
I20260809 14:30:19.952979 32476 rpcz_store.cc:276] 0809 14:30:18.454270 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.454523 (+ 253us) service_pool.cc:227] Handling call
0809 14:30:18.702266 (+247743us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b5800001898. stack: 00007f555ab48033 00007f555ab47f30 00007f555a6c9534 00000000004db2b4 00000000004db249 00007f555a9bef11 00007f555ab42851 00007f555ab42702 00007f55639d9b09 00007f5562a36730 00007f5562b18d21 00007f5562af61e3 00007f5562aa7869 00007f5561199c44 00007f5561199bd2 00007f5561199b01
0809 14:30:18.702464 (+ 198us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.702642 (+ 178us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.338596 (+635954us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.792166 (+453570us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.952100 (+159934us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":61852,"spinlock_wait_cycles":11880192}
I20260809 14:30:19.958032 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 253) took 2744 ms. Trace:
I20260809 14:30:19.958437 996 rpcz_store.cc:276] 0809 14:30:17.213515 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.213691 (+ 176us) service_pool.cc:227] Handling call
0809 14:30:19.413134 (+2199443us) ts_tablet_manager.cc:1145] Deleting tablet 94ff6ff27f6144d08d68a5e79c12963f
0809 14:30:19.413195 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:19.654167 (+240972us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.904960 (+250793us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.957699 (+ 52739us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":49867,"tablet-delete.queue_time_us":2198773}
I20260809 14:30:19.959203 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 7 candidate_status { last_received { term: 6 index: 320 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:30:19.959841 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 94ff6ff27f6144d08d68a5e79c12963f (table default.Table_e58ccd00bec645e4be0b92281bbcc459 [id=b17df3051f7d4322a4cf491f54bd7019]) successfully deleted
I20260809 14:30:19.961057 996 tablet_replica.cc:330] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:19.961699 996 raft_consensus.cc:2229] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:19.967808 711 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d10512ac8b3406098fad1f7da012ab3; no voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:30:19.969535 1010 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 14:30:19.982323 32475 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 286) took 1443 ms. Trace:
I20260809 14:30:19.985952 32470 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 289) took 1348 ms. Trace:
I20260809 14:30:19.986732 32470 rpcz_store.cc:276] 0809 14:30:18.637384 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.637830 (+ 446us) service_pool.cc:227] Handling call
0809 14:30:18.697057 (+ 59227us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.697226 (+ 169us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.468529 (+771303us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.984951 (+516422us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.985840 (+ 889us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41922}
I20260809 14:30:19.991297 32474 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 287) took 1448 ms. Trace:
I20260809 14:30:19.992061 32474 rpcz_store.cc:276] 0809 14:30:18.542869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.543278 (+ 409us) service_pool.cc:227] Handling call
0809 14:30:18.694568 (+151290us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.694766 (+ 198us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.679036 (+984270us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.990366 (+311330us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.991226 (+ 860us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11000,"spinlock_wait_cycles":32000}
I20260809 14:30:19.997777 799 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" instance_seqno: 1786285787196394) as {username='slave'} at 127.30.252.65:39679; Asking this server to re-register.
I20260809 14:30:20.012600 786 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" instance_seqno: 1786285791473234) as {username='slave'} at 127.30.252.67:53209; Asking this server to re-register.
W20260809 14:30:20.013984 786 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.67:53209 (request call id 10) took 1145 ms (client timeout 1000 ms). Trace:
W20260809 14:30:20.014741 786 rpcz_store.cc:269] 0809 14:30:18.868841 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.870184 (+ 1343us) service_pool.cc:227] Handling call
0809 14:30:20.013819 (+1143635us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:20.028791 1125 catalog_manager.cc:1259] Loaded cluster ID: f8f50e99f67f4e0a9d8c4cfa96e7ed70
I20260809 14:30:20.029353 1125 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: loading cluster ID for follower catalog manager: success
I20260809 14:30:19.997684 32479 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 293) took 1262 ms. Trace:
I20260809 14:30:20.000617 32482 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 295) took 1260 ms. Trace:
I20260809 14:30:20.001471 996 raft_consensus.cc:2258] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:19.837531 32705 consensus_queue.cc:260] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:20.080811 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 90e67f393f7a4bb987edc99aa86a9c90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:19 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:20.085063 32224 heartbeater.cc:464] Registering TS with master...
I20260809 14:30:20.076380 996 ts_tablet_manager.cc:1905] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:20.097528 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 947c54732ccc406b987b06eef17be366 (DEFAULT_TABLE table=default.Table_895a6d5bcb404eb8aaec0484b109e6aa [id=8db2c6da5cfb441082a17148c64a630a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.104686 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947c54732ccc406b987b06eef17be366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.108117 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:20.090000 32134 tablet_service.cc:1514] Processing DeleteTablet for tablet 666bea53f14541368730f69402c2b411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:19 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:20.123559 32439 leader_election.cc:341] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:20.126348 32487 tablet_service.cc:1467] Processing CreateTablet for tablet 060e3ed3be3140559a59f7204ce0a2b6 (DEFAULT_TABLE table=default.Table_089b0a96a71948bd8bc0df4624d303f2 [id=3016ad78c9fd4f8f89874c7da5dcf3c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.127327 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e3ed3be3140559a59f7204ce0a2b6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:20.131265 1149 log.cc:924] Time spent T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Append to log took a long time: real 0.298s user 0.001s sys 0.000s
I20260809 14:30:19.983069 32475 rpcz_store.cc:276] 0809 14:30:18.539015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.539429 (+ 414us) service_pool.cc:227] Handling call
0809 14:30:18.694316 (+154887us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.694462 (+ 146us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.443918 (+749456us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.981214 (+537296us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.982211 (+ 997us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12752,"spinlock_wait_cycles":9472}
W20260809 14:30:20.137849 32095 leader_election.cc:341] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 79bb4dd74afe4ace94eb1519950a83c0
I20260809 14:30:20.152966 32476 tablet_service.cc:1467] Processing CreateTablet for tablet 947c54732ccc406b987b06eef17be366 (DEFAULT_TABLE table=default.Table_895a6d5bcb404eb8aaec0484b109e6aa [id=8db2c6da5cfb441082a17148c64a630a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.150858 1125 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: acquiring CA information for follower catalog manager: success
I20260809 14:30:20.154145 32476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947c54732ccc406b987b06eef17be366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.157191 32470 tablet_service.cc:1467] Processing CreateTablet for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab (DEFAULT_TABLE table=default.Table_8405c747a7ce4e579d0fe4c17912d70b [id=896010e13dec44f58076ec15ecdbc58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.158317 32470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.117038 32479 rpcz_store.cc:276] 0809 14:30:18.734807 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.734963 (+ 156us) service_pool.cc:227] Handling call
0809 14:30:18.810497 (+ 75534us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.810638 (+ 141us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.556820 (+746182us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.996541 (+439721us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:19.997582 (+ 1041us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:20.076838 32482 rpcz_store.cc:276] 0809 14:30:18.739727 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.743283 (+ 3556us) service_pool.cc:227] Handling call
0809 14:30:18.816918 (+ 73635us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.828138 (+ 11220us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.620309 (+792171us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:19.999654 (+379345us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:20.000507 (+ 853us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:20.220652 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c2edbb238dd341b49bcd5acde282fb46" } modify_external_catalogs: true
I20260809 14:30:20.221187 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c2edbb238dd341b49bcd5acde282fb46" } modify_external_catalogs: true
I20260809 14:30:20.221560 31942 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_be768f7988814d00aeb5aea5c1b5c91c" } new_table_name: "default.Table_202a2a6d9c0b42c2940026fafa15fafe" modify_external_catalogs: true
W20260809 14:30:20.223603 31938 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 48ad4aee960b4ca5bebeab8ba4899c29 consensus state has no leader
I20260809 14:30:20.224647 31944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f13fa07f50084edc949a28f2c35bd624" } modify_external_catalogs: true
I20260809 14:30:20.225137 31944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_f13fa07f50084edc949a28f2c35bd624" } modify_external_catalogs: true
I20260809 14:30:20.243659 786 ts_manager.cc:194] Registered new tserver with Master: e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831)
I20260809 14:30:20.254395 1125 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 14:30:20.260120 32124 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:41528 (request call id 55) took 2401 ms. Trace:
I20260809 14:30:20.260617 32124 rpcz_store.cc:276] 0809 14:30:17.858872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.944065 (+ 85193us) service_pool.cc:227] Handling call
0809 14:30:18.282829 (+338764us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54000a3a54. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181471 00007fb6c71f6268 00007fb6c81e26cd 00007fb6c6a8ae04 00007fb6c6a8ad92 00007fb6c6a8acc1 00007fb6c6a8ac3c 00007fb6c6a89ef2 00007fb6c4fab94c 00007fb6c4faad86 00007fb6c4faa72f
0809 14:30:20.248330 (+1965501us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54000d2ad4. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181471 00007fb6c71f6268 00007fb6c81e26cd 00007fb6c6a8ae04 00007fb6c6a8ad92 00007fb6c6a8acc1 00007fb6c6a8ac3c 00007fb6c6a89ef2 00007fb6c4fab94c 00007fb6c4faad86 00007fb6c4faa72f
0809 14:30:20.260032 (+ 11702us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":781229184}
I20260809 14:30:20.264567 32133 tablet_service.cc:1467] Processing CreateTablet for tablet 41960774cfb647e4a7c4534e0afd332e (DEFAULT_TABLE table=default.Table_c5b668a1380d48a893831c7012ce015d [id=b379d99477a344928a6e1878803482b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.268697 1096 raft_consensus.cc:482] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:20.269192 1096 raft_consensus.cc:504] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:20.271248 1096 leader_election.cc:288] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:20.271478 1162 tablet.cc:1722] T ebc6ebda27b44048b5734e8798a980c2 P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:20.273227 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:20.291044 31937 master_service.cc:946] ReplaceTablet: received request to replace tablet 200c48c11cb042169b307bc1902be82e from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:20.291733 32482 tablet_service.cc:1467] Processing CreateTablet for tablet 41960774cfb647e4a7c4534e0afd332e (DEFAULT_TABLE table=default.Table_c5b668a1380d48a893831c7012ce015d [id=b379d99477a344928a6e1878803482b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.291953 1096 raft_consensus.cc:482] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:20.292510 1096 raft_consensus.cc:504] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:20.295884 1096 leader_election.cc:288] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:20.297672 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:20.303735 820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49178 (request call id 0) took 6367 ms. Trace:
I20260809 14:30:20.308355 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 6812d380b8d143a0bc35c4dc9b2aaeeb (DEFAULT_TABLE table=default.Table_35322bfb7f9a4e28a3ca0725dedb6cf3 [id=cc89bf16f3584485beb880290cd29b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.320401 32691 tablet_bootstrap.cc:492] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:20.320887 32691 ts_tablet_manager.cc:1397] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 1.421s user 0.022s sys 0.011s
W20260809 14:30:20.327632 31735 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 47335.2us
I20260809 14:30:20.341831 1161 tablet.cc:1722] T ebc6ebda27b44048b5734e8798a980c2 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:20.361181 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41960774cfb647e4a7c4534e0afd332e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.382995 32479 tablet_service.cc:1467] Processing CreateTablet for tablet 6812d380b8d143a0bc35c4dc9b2aaeeb (DEFAULT_TABLE table=default.Table_35322bfb7f9a4e28a3ca0725dedb6cf3 [id=cc89bf16f3584485beb880290cd29b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.385188 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6812d380b8d143a0bc35c4dc9b2aaeeb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:20.326316 1160 log.cc:924] Time spent T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
I20260809 14:30:20.326838 32705 ts_tablet_manager.cc:1428] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.576s user 0.021s sys 0.018s
W20260809 14:30:20.388792 32705 ts_tablet_manager.cc:1460] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb: Tablet startup took 638ms
W20260809 14:30:20.389076 32705 ts_tablet_manager.cc:1462] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb: Trace:
0809 14:30:01.552260 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.556809 (+ 4549us) service_pool.cc:227] Handling call
0809 14:30:01.558346 (+ 1537us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:01.558521 (+ 175us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.620805 (+ 62284us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.252064 (+631259us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.252656 (+ 592us) inbound_call.cc:173] Queueing success response
0809 14:30:19.072341 (+16819685us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 14:30:19.750533 (+678192us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 14:30:19.807653 (+ 57120us) tablet_replica.cc:259] Starting instrumentation
0809 14:30:19.816189 (+ 8536us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":22329,"raft.queue_time_us":64232,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":173,"tablet-open.queue_time_us":16819758,"thread_start_us":183225,"threads_started":1}
I20260809 14:30:20.427536 1162 tablet.cc:1722] T 2a689385d063469595fcae82aa979bb1 P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 14:30:20.439985 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_1c716de0a0fa40f7aa65ec405fceac00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:20.440654 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_753e14e7053644a3a98b60a13a6c9a66" } new_table_name: "default.Table_6e1851b65fd34b339ca4fab8be98c35e" modify_external_catalogs: true
W20260809 14:30:20.441460 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c716de0a0fa40f7aa65ec405fceac00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 14:30:20.454632 31942 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9d700191322341f998c5a8bfed609299 consensus state has no leader
I20260809 14:30:20.455463 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet b12d4849dec841a98e7f1a00c7d5f24e
I20260809 14:30:20.457195 32135 tablet_service.cc:1514] Processing DeleteTablet for tablet b12d4849dec841a98e7f1a00c7d5f24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:20.458246 32477 tablet_service.cc:1514] Processing DeleteTablet for tablet b12d4849dec841a98e7f1a00c7d5f24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:20.468647 1163 tablet.cc:1722] T 2a689385d063469595fcae82aa979bb1 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 14:30:20.491207 31944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 7ce3271ee8684662a9d3cd56157e572a
I20260809 14:30:20.493937 32477 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ce3271ee8684662a9d3cd56157e572a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:20.494073 32135 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ce3271ee8684662a9d3cd56157e572a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:20.496869 32133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41960774cfb647e4a7c4534e0afd332e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.506130 32705 tablet_bootstrap.cc:492] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:20.537477 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 200c48c11cb042169b307bc1902be82e
I20260809 14:30:20.532807 1086 raft_consensus.cc:482] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:20.538969 31937 catalog_manager.cc:6565] ReplaceTablet: tablet 200c48c11cb042169b307bc1902be82e deleted and replaced by tablet 64ce0263e6e7484dac027da29836b74e
W20260809 14:30:20.539145 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23aa8db34e704f7aa56acb6af46be5bc consensus state has no leader
I20260809 14:30:20.537993 1086 raft_consensus.cc:504] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:20.540618 32135 tablet_service.cc:1514] Processing DeleteTablet for tablet 200c48c11cb042169b307bc1902be82e with delete_type TABLET_DATA_DELETED (replaced by tablet 64ce0263e6e7484dac027da29836b74e) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:20.553723 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6812d380b8d143a0bc35c4dc9b2aaeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.506927 32691 raft_consensus.cc:348] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:19.268179 32260 leader_election.cc:302] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
W20260809 14:30:20.561463 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 1.29314e+06us
I20260809 14:30:20.585366 31943 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd] alter complete (version 1)
I20260809 14:30:20.591078 786 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" instance_seqno: 1786285791473234) as {username='slave'} at 127.30.252.67:53209; Asking this server to re-register.
I20260809 14:30:20.583945 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf222c49c934d52a2642f20e8575702" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:20.606194 32299 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 277) took 1364 ms. Trace:
I20260809 14:30:20.606541 32299 rpcz_store.cc:276] 0809 14:30:19.241275 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:19.241383 (+ 108us) service_pool.cc:227] Handling call
0809 14:30:19.281190 (+ 39807us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:19.281693 (+ 503us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:20.566540 (+1284847us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.605317 (+ 38777us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:20.605954 (+ 637us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:20.546458 1161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:44626 (request call id 303) took 1064 ms. Trace:
W20260809 14:30:20.609659 32439 leader_election.cc:341] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:20.609632 1161 rpcz_store.cc:276] 0809 14:30:19.482377 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:19.484716 (+ 2339us) service_pool.cc:227] Handling call
0809 14:30:20.546384 (+1061668us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:19.667196 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:19.756852 (+ 89656us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:19.756885 (+ 33us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:19.756937 (+ 52us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:19.757073 (+ 136us) alter_schema_op.cc:126] START. Timestamp: 1786285819747517
0809 14:30:19.757105 (+ 32us) op_driver.cc:331] REPLICATION: starting
0809 14:30:19.757887 (+ 782us) log.cc:842] Serialized 173 byte log entry
0809 14:30:20.267549 (+509662us) op_driver.cc:444] REPLICATION: finished
0809 14:30:20.341589 (+ 74040us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:20.344235 (+ 2646us) log.cc:842] Serialized 5 byte log entry
0809 14:30:20.508281 (+164046us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.532604 (+ 24323us) log.cc:842] Serialized 14 byte log entry
0809 14:30:20.546198 (+ 13594us) alter_schema_op.cc:71] Released schema lock
0809 14:30:20.546238 (+ 40us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":73775,"mutex_wait_us":34513,"prepare.queue_time_us":160737,"prepare.run_cpu_time_us":4452,"prepare.run_wall_time_us":243101,"raft.queue_time_us":92496,"raft.run_cpu_time_us":1705,"raft.run_wall_time_us":1705,"replication_time_us":510358,"thread_start_us":274483,"threads_started":3,"wal-append.queue_time_us":72348}]]}
I20260809 14:30:20.545720 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 200c48c11cb042169b307bc1902be82e with delete_type TABLET_DATA_DELETED (replaced by tablet 64ce0263e6e7484dac027da29836b74e) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:20.614238 32691 raft_consensus.cc:374] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:20.619266 31944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_4878b3fdb4ad4a479d83e997033a54ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:20.620930 31944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4878b3fdb4ad4a479d83e997033a54ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:20.623272 31943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9f57196e69e94c7c8e1f34e2b75a48c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:20.623821 1156 raft_consensus.cc:482] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:20.624293 1156 raft_consensus.cc:504] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:20.625573 31943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f57196e69e94c7c8e1f34e2b75a48c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:20.626176 32163 raft_consensus.cc:3037] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:20.626645 1156 leader_election.cc:288] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:20.597278 1004 ts_tablet_manager.cc:1918] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:20.630236 32691 raft_consensus.cc:729] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
W20260809 14:30:20.616878 32480 consensus_meta.cc:287] T d2da78cb058d400495986a23be4b5d56 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent flushing consensus metadata: real 0.511s user 0.006s sys 0.000s
I20260809 14:30:20.639153 32509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:20.646656 32691 consensus_queue.cc:260] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:20.657661 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_5850a142ddb549ce82ca00ddc814fcbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:20.660084 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_4b848656d0ca4f2b81bd9e56d4540967" } new_table_name: "default.Table_1ce1e7baa3ab4b38a15aa6009f3dd65c" modify_external_catalogs: true
I20260809 14:30:20.662608 31937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ed4114d732c44266bc8e100a01ce3499" } modify_external_catalogs: true
I20260809 14:30:20.662928 1167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:30:20.663662 31937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_ed4114d732c44266bc8e100a01ce3499" } modify_external_catalogs: true
I20260809 14:30:20.663394 1167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:20.664793 32146 tablet_service.cc:1467] Processing CreateTablet for tablet f139a623619842418f7cf3db3675b5da (DEFAULT_TABLE table=default.Table_82a85f2bf88341b0aa7081c4484a2817 [id=9465fb3db77849e89fb1075b4f86ac40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.668715 31869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 7 candidate_status { last_received { term: 6 index: 320 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
W20260809 14:30:20.659394 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5850a142ddb549ce82ca00ddc814fcbf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:20.644058 1086 leader_election.cc:288] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:20.588192 32301 consensus_meta.cc:287] T 43931b0436c4472eb12ee1afd46c1900 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 1.224s user 0.005s sys 0.005s
I20260809 14:30:20.682889 32482 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee59b2efd4f4c6c9c8f58fee02c4fd7 (DEFAULT_TABLE table=default.Table_a3796b4fbd05463eb6193485302bcd5e [id=a9b5a8f50bb440be853d4e6dca619354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.684350 32480 tablet_service.cc:1467] Processing CreateTablet for tablet f139a623619842418f7cf3db3675b5da (DEFAULT_TABLE table=default.Table_82a85f2bf88341b0aa7081c4484a2817 [id=9465fb3db77849e89fb1075b4f86ac40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.686244 32135 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee59b2efd4f4c6c9c8f58fee02c4fd7 (DEFAULT_TABLE table=default.Table_a3796b4fbd05463eb6193485302bcd5e [id=a9b5a8f50bb440be853d4e6dca619354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.689841 32691 ts_tablet_manager.cc:1428] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.369s user 0.007s sys 0.001s
I20260809 14:30:20.702598 32569 heartbeater.cc:464] Registering TS with master...
I20260809 14:30:20.703378 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet 2af4e169f518447c8d29c630dca93ff9
I20260809 14:30:20.705400 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee59b2efd4f4c6c9c8f58fee02c4fd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.709659 31945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_424b18fbca56430bb3e23219c6153e97" } modify_external_catalogs: true
I20260809 14:30:20.710201 31945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_424b18fbca56430bb3e23219c6153e97" } modify_external_catalogs: true
I20260809 14:30:20.711320 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af4e169f518447c8d29c630dca93ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:20.670862 1004 log.cc:1196] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/2f171260cb764ac284aef85e4219767e
I20260809 14:30:20.717896 32705 tablet_bootstrap.cc:654] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:20.718562 1004 ts_tablet_manager.cc:1939] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
W20260809 14:30:20.735029 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3d81046d929c48a3a6946f20d8fd7bca consensus state has no leader
I20260809 14:30:20.748775 1009 raft_consensus.cc:2781] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:20.749255 1009 raft_consensus.cc:482] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:20.749600 1009 raft_consensus.cc:3037] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:20.755225 1020 ts_tablet_manager.cc:1905] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:20.759092 31948 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_bc907356c8584717b336afc1a260018e" } new_table_name: "default.Table_8417c7eb8d6b468b8af035d73ac48bc7" modify_external_catalogs: true
W20260809 14:30:20.764631 32304 consensus_meta.cc:287] T 0b476fe571374e33a3b4d11057c713ea P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 1.188s user 0.009s sys 0.000s
I20260809 14:30:20.765636 32304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 273) took 1809 ms. Trace:
I20260809 14:30:20.765913 32304 rpcz_store.cc:276] 0809 14:30:18.961365 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.980278 (+ 18913us) service_pool.cc:227] Handling call
0809 14:30:19.012670 (+ 32392us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:19.012827 (+ 157us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.554597 (+541770us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.765130 (+1210533us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:20.765591 (+ 461us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 14:30:20.768868 32303 consensus_meta.cc:287] T 514327321bf142379e8103a4b8c1a937 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 1.340s user 0.010s sys 0.000s
I20260809 14:30:20.769840 32303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 275) took 1770 ms. Trace:
I20260809 14:30:20.772032 31942 catalog_manager.cc:6040] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: default.Table_cad12e9e9662472d80abf06e813261b3 [id=c19378f8e8994ef28e7b0eba079948c3] alter complete (version 3)
I20260809 14:30:20.770134 32303 rpcz_store.cc:276] 0809 14:30:18.999981 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:19.000202 (+ 221us) service_pool.cc:227] Handling call
0809 14:30:19.001524 (+ 1322us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:19.001615 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.427616 (+426001us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.769328 (+1341712us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:20.769793 (+ 465us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 14:30:20.781181 32693 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 1.085s user 0.006s sys 0.004s
W20260809 14:30:20.781481 32693 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 1.124s user 0.006s sys 0.004s
I20260809 14:30:20.783742 32693 tablet_bootstrap.cc:492] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:20.784214 32693 ts_tablet_manager.cc:1397] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 1.439s user 0.007s sys 0.017s
I20260809 14:30:20.787034 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_9527df93a9844cf2b0b2cbe5a98d6cae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:20.788564 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9527df93a9844cf2b0b2cbe5a98d6cae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:20.798177 32307 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee6e4c5bf5a464589fcfe3b39220a8b (DEFAULT_TABLE table=default.Table_61526cd7c2b14f0a916e4d916c607a78 [id=c121698e43f84d8eae0939cfec3a36cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.800093 32304 tablet_service.cc:1467] Processing CreateTablet for tablet 2727d6a119c248c8aeb7dd25b7a8ccda (DEFAULT_TABLE table=default.Table_0e45100c9aee47e5a5a225d0f0564718 [id=c34ab4b002be4a2c8effce6c44952ebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.800555 32295 tablet_service.cc:1467] Processing CreateTablet for tablet 7b7decba4a9a45f1b3e5bb21c54c924e (DEFAULT_TABLE table=default.Table_971214cd6b5048afb1c306f0b0c55219 [id=261eb132302e4d74bb18d23eea8ee9d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.800985 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet ea0a36a6ddc64557b31d0dfef8b5b457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:20.802392 32308 tablet_service.cc:1467] Processing CreateTablet for tablet a9c969f1ec4744fdb8b810793b099cc8 (DEFAULT_TABLE table=default.Table_7f6aaf330b564e2c8a3126bca9401698 [id=567557fc695846c0aa78b86becb353bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.804015 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 370a8a83f0c840c1b08d92188e5b62d9 (DEFAULT_TABLE table=default.Table_ac82ee262e4a40bcb9bc60cc74a8861f [id=65aaec21b1e5496a9b094e61bbd4f46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.806169 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 48ac456e1a2c48c9851fb139c02ac40b (DEFAULT_TABLE table=default.Table_01106b7a16e142518d374e93aac4da95 [id=f5cc092693d34d67a8a27b147fa767ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.816035 32298 tablet_service.cc:1514] Processing DeleteTablet for tablet f74b4deeb32941aeaa3c4b813333c14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:20.821652 996 ts_tablet_manager.cc:1918] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:20.821990 996 log.cc:1196] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/4cdfc56edcd34696a7b806b0ee34d594
I20260809 14:30:20.822608 996 ts_tablet_manager.cc:1939] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:20.825448 32145 tablet_service.cc:1467] Processing CreateTablet for tablet b526764f8d2b43d6942167ffa5d34721 (DEFAULT_TABLE table=default.Table_f01ac5f2b8b54a57a444f006d814b132 [id=9ffc3631d1aa48a88e6aebbf24d9217b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.826310 799 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" instance_seqno: 1786285789236264) as {username='slave'} at 127.30.252.66:51593; Asking this server to re-register.
I20260809 14:30:20.828400 32134 tablet_service.cc:1467] Processing CreateTablet for tablet ba12d9ba8c0e47879713159348ba1126 (DEFAULT_TABLE table=default.Table_fe15193782d744c19d096d034729510a [id=05974613d5a54b60b00f6f292ec483e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.829229 32298 tablet_service.cc:1467] Processing CreateTablet for tablet 401552b9d283466fbfeaf9ad8ed0602f (DEFAULT_TABLE table=default.Table_d7a3df21ac0a4fd2b7cb295edfb62f96 [id=217270d577db418e87628191169460be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.818336 32303 tablet_service.cc:1514] Processing DeleteTablet for tablet 239ad14cb3a642e08bace0759bc0b732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:20.831910 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 7 candidate_status { last_received { term: 6 index: 320 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:30:20.833578 711 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d10512ac8b3406098fad1f7da012ab3; no voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:30:20.836158 1167 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:30:20.838601 1167 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 14:30:20.846185 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f139a623619842418f7cf3db3675b5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.845750 1009 raft_consensus.cc:504] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:20.847877 1009 leader_election.cc:288] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:20.850082 32691 tablet_bootstrap.cc:492] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:20.850728 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee6e4c5bf5a464589fcfe3b39220a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.856985 32300 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 276) took 1751 ms. Trace:
I20260809 14:30:20.857473 32300 rpcz_store.cc:276] 0809 14:30:19.105325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:19.111439 (+ 6114us) service_pool.cc:227] Handling call
0809 14:30:19.267085 (+155646us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:19.267265 (+ 180us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:20.595106 (+1327841us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.805209 (+210103us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:20.856797 (+ 51588us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:20.866086 32300 tablet_service.cc:1467] Processing CreateTablet for tablet 59134b4266764866aca8030f9240e3e7 (DEFAULT_TABLE table=default.Table_1474c6854a634c82909f0cc97490d2ea [id=9441e797a40845cc8376f490da7f57df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.867198 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59134b4266764866aca8030f9240e3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.867985 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 256) took 3620 ms. Trace:
I20260809 14:30:20.868690 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 35317d7ca48544fda59a4ca273406814 (DEFAULT_TABLE table=default.Table_b62ea93ef63a4b0ba177bdae01cc95b4 [id=9618dc012e8e47c08511b9bf13480892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.868305 996 rpcz_store.cc:276] 0809 14:30:17.247908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.248037 (+ 129us) service_pool.cc:227] Handling call
0809 14:30:19.960902 (+2712865us) ts_tablet_manager.cc:1145] Deleting tablet 4cdfc56edcd34696a7b806b0ee34d594
0809 14:30:19.960951 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:20.377804 (+416853us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.821431 (+443627us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.867921 (+ 46490us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":39279,"tablet-delete.queue_time_us":2712315}
I20260809 14:30:20.870633 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b7decba4a9a45f1b3e5bb21c54c924e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.871083 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 267) took 3452 ms. Trace:
I20260809 14:30:20.871407 996 rpcz_store.cc:276] 0809 14:30:17.418518 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.418747 (+ 229us) service_pool.cc:227] Handling call
0809 14:30:20.870760 (+3452013us) ts_tablet_manager.cc:1145] Deleting tablet 9e38a13e6e1c480880eab4bc2d2113cd
0809 14:30:20.870801 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:20.871034 (+ 233us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3450506}
I20260809 14:30:20.872334 996 tablet_replica.cc:330] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:20.872968 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ac456e1a2c48c9851fb139c02ac40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.873102 996 raft_consensus.cc:2229] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:20.873478 996 pending_rounds.cc:62] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Trying to abort 1 pending ops.
I20260809 14:30:20.873669 996 pending_rounds.cc:68] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316626704834408448 op_type: NO_OP noop_request { }
I20260809 14:30:20.873989 996 raft_consensus.cc:2866] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 14:30:20.874305 996 raft_consensus.cc:2258] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:20.877029 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370a8a83f0c840c1b08d92188e5b62d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.880976 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c969f1ec4744fdb8b810793b099cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.881563 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2727d6a119c248c8aeb7dd25b7a8ccda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.882088 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35317d7ca48544fda59a4ca273406814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.882577 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401552b9d283466fbfeaf9ad8ed0602f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.890992 32691 tablet_bootstrap.cc:654] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:20.895071 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:44626 (request call id 302) took 1933 ms. Trace:
I20260809 14:30:20.902848 31939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_11709a3dc2554d2f98756c54dba95747"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 14:30:20.904032 32693 raft_consensus.cc:348] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:20.904561 31939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11709a3dc2554d2f98756c54dba95747 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:20.904740 32693 raft_consensus.cc:374] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:20.905026 32693 raft_consensus.cc:729] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:20.905614 32693 consensus_queue.cc:260] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:20.909811 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_42a687657ecc485a9c3d956e245784c5" } modify_external_catalogs: true
I20260809 14:30:20.910313 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_42a687657ecc485a9c3d956e245784c5" } modify_external_catalogs: true
I20260809 14:30:20.912734 1169 raft_consensus.cc:482] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:20.913219 1169 raft_consensus.cc:504] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:20.915119 1169 leader_election.cc:288] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:20.925695 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f139a623619842418f7cf3db3675b5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.927032 31945 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 8b272af7a99e46d38f3303682a1b03a9
I20260809 14:30:20.915762 32301 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 274) took 1932 ms. Trace:
I20260809 14:30:20.928639 32301 rpcz_store.cc:276] 0809 14:30:18.983118 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.996517 (+ 13399us) service_pool.cc:227] Handling call
0809 14:30:18.997920 (+ 1403us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.997988 (+ 68us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:19.363807 (+365819us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.817120 (+1453313us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:20.915711 (+ 98591us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:20.930351 32303 tablet_service.cc:1467] Processing CreateTablet for tablet 11f5459874e841e2b39a0e2c5f4d2f87 (DEFAULT_TABLE table=default.Table_d50800cad30d400ba6b4bb95b5f60042 [id=b8301d685310445bbdc4b58b9f3a6e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.931452 32303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11f5459874e841e2b39a0e2c5f4d2f87. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:20.936051 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 60acf8a11d6143948bfe0040f74f5ef8 (table default.Table_6d1220a2972747a7b2050e1714d86647 [id=217a91e1a58b443d93738772edc80b76], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 60acf8a11d6143948bfe0040f74f5ef8 consensus state has no leader
I20260809 14:30:20.936903 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 36350e37e5834491a352417543f7564c (DEFAULT_TABLE table=default.Table_0227da888bb24888976eae18fce543e9 [id=bb9c3a0b52f0419db0b22211334a0aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.942158 32141 tablet_service.cc:1467] Processing CreateTablet for tablet d10a78dcb5c94435a12fb4d87b63b1a8 (DEFAULT_TABLE table=default.Table_b27d4b6a8d4d47388f293a343c5d11ed [id=8f6e37550cfa41609d2add1f73f3790e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.946089 32310 tablet_service.cc:1467] Processing CreateTablet for tablet a1bfe0fa1bb246ffb434b7aa9652935f (DEFAULT_TABLE table=default.Table_b219214ad9f94dd991137e6a5a35764c [id=7b60c14a5a764cb9abae551857fb55a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.947099 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1bfe0fa1bb246ffb434b7aa9652935f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.950430 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee59b2efd4f4c6c9c8f58fee02c4fd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.955927 31945 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b219214ad9f94dd991137e6a5a35764c" } new_table_name: "default.Table_229f88bff6c54d5186d3aa8190d0ef9e" modify_external_catalogs: true
I20260809 14:30:20.957041 32145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b526764f8d2b43d6942167ffa5d34721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.960992 32134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba12d9ba8c0e47879713159348ba1126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.964998 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36350e37e5834491a352417543f7564c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.966130 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10a78dcb5c94435a12fb4d87b63b1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:20.969933 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 277) took 3118 ms. Trace:
I20260809 14:30:20.971553 1004 rpcz_store.cc:276] 0809 14:30:17.851359 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.854677 (+ 3318us) service_pool.cc:227] Handling call
0809 14:30:19.516224 (+1661547us) ts_tablet_manager.cc:1145] Deleting tablet 2f171260cb764ac284aef85e4219767e
0809 14:30:19.516272 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:20.298189 (+781917us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.597053 (+298864us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.969810 (+372757us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1660898}
I20260809 14:30:20.973129 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 5b399c4ca4b741098b6ab52bf9cc3e48 (DEFAULT_TABLE table=default.Table_f4339baad1a14705a1f3f35db5081a8b [id=6c55b37d949a4648bd2d31852bb7727d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:20.974202 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b399c4ca4b741098b6ab52bf9cc3e48. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:20.979200 31948 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930fb8d5f682462183c780ab52ce6dd6 consensus state has no leader
I20260809 14:30:20.909879 996 ts_tablet_manager.cc:1905] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:21.001647 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 275) took 3151 ms. Trace:
I20260809 14:30:21.002333 1004 rpcz_store.cc:276] 0809 14:30:17.850662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.855397 (+ 4735us) service_pool.cc:227] Handling call
0809 14:30:21.001135 (+3145738us) ts_tablet_manager.cc:1145] Deleting tablet b0ad3946f5de4387af3627a88b6404cf
0809 14:30:21.001228 (+ 93us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:21.001550 (+ 322us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3145187}
I20260809 14:30:21.003705 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 2f171260cb764ac284aef85e4219767e (table default.Table_42864987ccd14c05a6909a6e1527e208 [id=032f6264a2c842f1ba6b007331ccfcc6]) successfully deleted
I20260809 14:30:21.004107 1004 tablet_replica.cc:330] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
W20260809 14:30:21.006003 31930 catalog_manager.cc:4808] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet b0ad3946f5de4387af3627a88b6404cf because the tablet was not found. No further retry: Not found: Tablet not found: b0ad3946f5de4387af3627a88b6404cf
I20260809 14:30:21.017616 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 408ee42ea5844741b9ab514cbe6a94e4 (DEFAULT_TABLE table=default.Table_fc6f88dc00b54ceab7215561f2a37bf9 [id=fbabe9cb4a6c4c618be1395858155c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.018811 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408ee42ea5844741b9ab514cbe6a94e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.016000 1163 rpcz_store.cc:276] 0809 14:30:18.962017 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.963276 (+ 1259us) service_pool.cc:227] Handling call
0809 14:30:20.895000 (+1931724us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:19.470279 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:19.471050 (+ 771us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:19.471117 (+ 67us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:19.483002 (+ 11885us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:19.483183 (+ 181us) alter_schema_op.cc:126] START. Timestamp: 1786285819473616
0809 14:30:19.483227 (+ 44us) op_driver.cc:331] REPLICATION: starting
0809 14:30:19.542094 (+ 58867us) log.cc:842] Serialized 173 byte log entry
0809 14:30:20.397250 (+855156us) op_driver.cc:444] REPLICATION: finished
0809 14:30:20.468546 (+ 71296us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:20.469929 (+ 1383us) log.cc:842] Serialized 5 byte log entry
0809 14:30:20.894026 (+424097us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.894785 (+ 759us) log.cc:842] Serialized 14 byte log entry
0809 14:30:20.894880 (+ 95us) alter_schema_op.cc:71] Released schema lock
0809 14:30:20.894901 (+ 21us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":71039,"prepare.queue_time_us":182340,"prepare.run_cpu_time_us":9787,"prepare.run_wall_time_us":306955,"replication_time_us":913888,"thread_start_us":325935,"threads_started":3,"wal-append.queue_time_us":220416}]]}
I20260809 14:30:21.024292 32147 tablet_service.cc:1467] Processing CreateTablet for tablet f9be01cb6d944a04992da23002688d91 (DEFAULT_TABLE table=default.Table_153b78d9a7934c7ca382c10459415af0 [id=6a339b319fcc44de9e1a5feae512ba05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.025540 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9be01cb6d944a04992da23002688d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.041700 32305 tablet_service.cc:1514] Processing DeleteTablet for tablet 04b74ba6259e47569d13dfe62d14a4af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:21.046867 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_eb4a12a97b084ef2963e4341cb88fb7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:21.049247 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb4a12a97b084ef2963e4341cb88fb7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:21.064589 32295 tablet_service.cc:1514] Processing DeleteTablet for tablet c4f4c1eb916c4b51a5e176d541da8574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:18 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:21.066164 32308 tablet_service.cc:1467] Processing CreateTablet for tablet e76a109e93dd47b18e4f437b3965c44c (DEFAULT_TABLE table=default.Table_46cf63f7388349388957842c51da95d6 [id=33a64d0e281e4939bef7f46be25aef43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.067408 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e76a109e93dd47b18e4f437b3965c44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.069033 32297 tablet_service.cc:1467] Processing CreateTablet for tablet d2da78cb058d400495986a23be4b5d56 (DEFAULT_TABLE table=default.Table_6d21f9a0c69445bfb2882750ba0792dc [id=ac6ecfcb8fe04daaa87fdf49567041af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.069896 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2da78cb058d400495986a23be4b5d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.099660 32693 ts_tablet_manager.cc:1428] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.315s user 0.008s sys 0.000s
I20260809 14:30:21.100641 32693 tablet_bootstrap.cc:492] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:21.109169 1004 raft_consensus.cc:2229] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:21.111184 1004 raft_consensus.cc:2258] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:21.113736 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_c379e2bf47a04cffbb6c3b0ec14c7a55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:21.115249 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c379e2bf47a04cffbb6c3b0ec14c7a55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:21.126683 1009 raft_consensus.cc:482] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bed14aff53a41b2bbf5a13ce55dec5a)
I20260809 14:30:21.127161 1009 raft_consensus.cc:504] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:21.130795 31937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8405c747a7ce4e579d0fe4c17912d70b" } modify_external_catalogs: true
I20260809 14:30:21.131332 31937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_8405c747a7ce4e579d0fe4c17912d70b" } modify_external_catalogs: true
I20260809 14:30:21.136240 32140 tablet_service.cc:1467] Processing CreateTablet for tablet 93514283ff8e4fad8dae1b76d8efd4c2 (DEFAULT_TABLE table=default.Table_82b4d1f100df47fd8c1c0fed677e4a5b [id=b26f9932b01e4616a9a4f5c96c77385f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.142122 32132 tablet_service.cc:1467] Processing CreateTablet for tablet f8ecf7edec2b48dcbaea4f1413b44f95 (DEFAULT_TABLE table=default.Table_558c3b629f83480ea933b6c8d7ee9d1d [id=7c797610f85a4da59bc2c26738f3f779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.144348 32705 tablet_bootstrap.cc:492] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:21.144780 32705 ts_tablet_manager.cc:1397] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.639s user 0.012s sys 0.007s
I20260809 14:30:21.149574 1009 leader_election.cc:288] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:21.157617 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 66504us
W20260809 14:30:21.164192 32260 leader_election.cc:341] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:21.198869 31944 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_c99fa72bd1bd441aaa57d3f32a1d1d9e" } new_table_name: "default.Table_76435d57443044e89e81a802eb4853b3" modify_external_catalogs: true
I20260809 14:30:21.165982 32296 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 272) took 2299 ms. Trace:
I20260809 14:30:21.165220 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdfc56edcd34696a7b806b0ee34d594" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:21.224722 32295 tablet_service.cc:1467] Processing CreateTablet for tablet 060e3ed3be3140559a59f7204ce0a2b6 (DEFAULT_TABLE table=default.Table_089b0a96a71948bd8bc0df4624d303f2 [id=3016ad78c9fd4f8f89874c7da5dcf3c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.242147 32139 tablet_service.cc:1467] Processing CreateTablet for tablet f6682dd422794fdd85ed33002db448d5 (DEFAULT_TABLE table=default.Table_348f7a78a51f40369f9fbd23580ff6b1 [id=667197f2e47b4910b1d8b1d3dd7f955d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.249035 32140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93514283ff8e4fad8dae1b76d8efd4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.249632 32132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ecf7edec2b48dcbaea4f1413b44f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.269287 1004 ts_tablet_manager.cc:1905] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:21.278206 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce9848ee8f7d45cba615b06b9c826afb" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:21.235816 32296 rpcz_store.cc:276] 0809 14:30:18.866180 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.866289 (+ 109us) service_pool.cc:227] Handling call
0809 14:30:18.957576 (+ 91287us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:18.957674 (+ 98us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:20.583918 (+1626244us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.107679 (+523761us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:21.165251 (+ 57572us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 14:30:21.286253 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 122084us
I20260809 14:30:21.290153 32304 tablet_service.cc:1467] Processing CreateTablet for tablet 48ad4aee960b4ca5bebeab8ba4899c29 (DEFAULT_TABLE table=default.Table_3344e8d1d6c347aaa907473e456adb3b [id=02a21b34d14e429d954e11d78733643f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.299296 32135 tablet_service.cc:1467] Processing CreateTablet for tablet a6d2c214e3e74a2db41f28f84ea4a65c (DEFAULT_TABLE table=default.Table_cad4e455f76b418f90ffde8a67a9566f [id=96e505caf4234af6942276b524209971]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.305477 32476 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af4e169f518447c8d29c630dca93ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:21.307492 32469 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44626 (request call id 304) took 1608 ms. Trace:
I20260809 14:30:21.308003 32469 rpcz_store.cc:276] 0809 14:30:19.698928 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:19.758497 (+ 59569us) service_pool.cc:227] Handling call
0809 14:30:19.932843 (+174346us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:19.933028 (+ 185us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:20.350479 (+417451us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:20.691965 (+341486us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:21.307196 (+615231us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:21.310937 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet a0b196e429514cbc803e9a99a986d4d2
I20260809 14:30:21.312487 32145 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b196e429514cbc803e9a99a986d4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:21.313822 31945 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1bfe0fa1bb246ffb434b7aa9652935f consensus state has no leader
I20260809 14:30:21.330205 31940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_04910b92ceac48bc8e23618fa93fc47a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:21.331899 31940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04910b92ceac48bc8e23618fa93fc47a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:21.370332 996 ts_tablet_manager.cc:1918] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:21.371071 996 log.cc:1196] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/e74db9caa7f94059b1bba4a082fe57ef
I20260809 14:30:21.371312 1192 raft_consensus.cc:482] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:21.372363 996 ts_tablet_manager.cc:1939] T e74db9caa7f94059b1bba4a082fe57ef P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:21.377583 32691 tablet_bootstrap.cc:492] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:21.378396 32691 ts_tablet_manager.cc:1397] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.528s user 0.012s sys 0.004s
I20260809 14:30:21.418279 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49782:
name: "default.Table_81a13b65f8b44af2b23390a495a93b06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 14:30:21.419859 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81a13b65f8b44af2b23390a495a93b06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 14:30:21.420121 1194 raft_consensus.cc:482] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:21.420796 1194 raft_consensus.cc:504] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.423609 1194 leader_election.cc:288] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:21.372292 1192 raft_consensus.cc:504] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.424171 1194 raft_consensus.cc:482] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.480418 1194 raft_consensus.cc:504] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.481563 1192 leader_election.cc:288] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:21.482522 1194 leader_election.cc:288] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:21.483848 32691 raft_consensus.cc:348] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.484577 32691 raft_consensus.cc:374] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:21.484815 32691 raft_consensus.cc:729] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:21.485419 32691 consensus_queue.cc:260] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.297356 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 90e67f393f7a4bb987edc99aa86a9c90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:19 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:21.486924 32691 ts_tablet_manager.cc:1428] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.108s user 0.009s sys 0.000s
I20260809 14:30:21.298714 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 947c54732ccc406b987b06eef17be366 (DEFAULT_TABLE table=default.Table_895a6d5bcb404eb8aaec0484b109e6aa [id=8db2c6da5cfb441082a17148c64a630a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.298962 32299 tablet_service.cc:1467] Processing CreateTablet for tablet 41960774cfb647e4a7c4534e0afd332e (DEFAULT_TABLE table=default.Table_c5b668a1380d48a893831c7012ce015d [id=b379d99477a344928a6e1878803482b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.361598 32296 tablet_service.cc:1467] Processing CreateTablet for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab (DEFAULT_TABLE table=default.Table_8405c747a7ce4e579d0fe4c17912d70b [id=896010e13dec44f58076ec15ecdbc58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.487991 32691 tablet_bootstrap.cc:492] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:21.388903 32307 tablet_service.cc:1514] Processing DeleteTablet for tablet 666bea53f14541368730f69402c2b411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:19 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:21.394951 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e3ed3be3140559a59f7204ce0a2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.479861 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 268) took 3510 ms. Trace:
I20260809 14:30:21.489279 996 rpcz_store.cc:276] 0809 14:30:17.968998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.978497 (+ 9499us) service_pool.cc:227] Handling call
0809 14:30:20.872198 (+2893701us) ts_tablet_manager.cc:1145] Deleting tablet e74db9caa7f94059b1bba4a082fe57ef
0809 14:30:20.872236 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:21.000664 (+128428us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.369680 (+369016us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.479775 (+110095us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2892982}
W20260809 14:30:21.490125 32443 leader_election.cc:341] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:21.490852 996 tablet_replica.cc:330] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:21.490607 32443 leader_election.cc:302] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:21.400753 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:21.492920 32326 raft_consensus.cc:2379] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6bed14aff53a41b2bbf5a13ce55dec5a in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:21.445032 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac7afd531a746cab7618ead98b4c4d9" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:21.449620 32310 tablet_service.cc:1467] Processing CreateTablet for tablet 6812d380b8d143a0bc35c4dc9b2aaeeb (DEFAULT_TABLE table=default.Table_35322bfb7f9a4e28a3ca0725dedb6cf3 [id=cc89bf16f3584485beb880290cd29b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.501367 32329 raft_consensus.cc:2454] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
W20260809 14:30:21.453282 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 44556.6us
W20260809 14:30:21.500524 32443 raft_consensus.cc:2670] T 4cdfc56edcd34696a7b806b0ee34d594 P 6bed14aff53a41b2bbf5a13ce55dec5a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:21.482129 32305 tablet_service.cc:1514] Processing DeleteTablet for tablet b12d4849dec841a98e7f1a00c7d5f24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:21.483976 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf222c49c934d52a2642f20e8575702" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:21.505365 32260 consensus_peers.cc:487] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:21.515388 31943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_bea4b6049d234a58a4bc8222fd20f462" } modify_external_catalogs: true
I20260809 14:30:21.518517 31938 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_b1f8694507e64827909363f73d0471b1" } new_table_name: "default.Table_0e899792451d4491a6b1fe8e30d1aafe" modify_external_catalogs: true
I20260809 14:30:21.533821 31937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab
W20260809 14:30:21.534795 31944 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 46d8d00bbf2e4f4289448e48536979ea consensus state has no leader
I20260809 14:30:21.538092 31943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49782:
table { table_name: "default.Table_bea4b6049d234a58a4bc8222fd20f462" } modify_external_catalogs: true
I20260809 14:30:21.549355 786 ts_manager.cc:194] Registered new tserver with Master: 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:21.545796 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 55865.4us
I20260809 14:30:21.559887 32691 tablet_bootstrap.cc:654] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:21.561360 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947c54732ccc406b987b06eef17be366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.565003 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41960774cfb647e4a7c4534e0afd332e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.579921 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:21 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:21.580935 32296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.586700 1156 raft_consensus.cc:482] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.587615 1156 raft_consensus.cc:504] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.592775 1156 leader_election.cc:288] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:21.594619 1156 raft_consensus.cc:482] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.609632 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6812d380b8d143a0bc35c4dc9b2aaeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.614396 1192 raft_consensus.cc:482] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.615468 1192 raft_consensus.cc:504] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.619807 1192 leader_election.cc:288] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:21.620818 1196 raft_consensus.cc:482] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.621670 1196 raft_consensus.cc:504] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.623651 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 4cdfc56edcd34696a7b806b0ee34d594 (table default.Table_16f3bdcb253e48bf892996d7b4fbee7f [id=2edc29b3b38c425a99df406bfc8cc24e]) successfully deleted
I20260809 14:30:21.626102 996 raft_consensus.cc:2229] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:21.626752 996 raft_consensus.cc:2258] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:21.626660 1196 leader_election.cc:288] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:21.627382 31930 catalog_manager.cc:4821] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): delete failed for tablet 9e38a13e6e1c480880eab4bc2d2113cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9e38a13e6e1c480880eab4bc2d2113cd already in progress: creating tablet
I20260809 14:30:21.630450 996 ts_tablet_manager.cc:1905] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:21.632448 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ad4aee960b4ca5bebeab8ba4899c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.634792 1196 raft_consensus.cc:482] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.635831 1196 raft_consensus.cc:504] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:21.639587 32443 leader_election.cc:341] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:21.639873 1196 leader_election.cc:288] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:21.640210 32443 leader_election.cc:302] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:21.643168 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6682dd422794fdd85ed33002db448d5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:21.643126 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 136745us
I20260809 14:30:21.643927 32135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d2c214e3e74a2db41f28f84ea4a65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:21.644381 1194 raft_consensus.cc:2726] T ce9848ee8f7d45cba615b06b9c826afb P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:21.645174 32260 consensus_peers.cc:487] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:21.647387 1192 raft_consensus.cc:482] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.648182 1192 raft_consensus.cc:504] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.652983 32705 raft_consensus.cc:348] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.653841 32705 raft_consensus.cc:374] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:21.654130 32705 raft_consensus.cc:729] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:21.662329 31729 master-stress-test.cc:578] Killing test thread 1/10
I20260809 14:30:21.673429 31729 master-stress-test.cc:580] Killed test thread 1/10
I20260809 14:30:21.673738 31729 master-stress-test.cc:578] Killing test thread 2/10
I20260809 14:30:21.680274 32307 tablet_service.cc:1467] Processing CreateTablet for tablet f139a623619842418f7cf3db3675b5da (DEFAULT_TABLE table=default.Table_82a85f2bf88341b0aa7081c4484a2817 [id=9465fb3db77849e89fb1075b4f86ac40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.681622 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f139a623619842418f7cf3db3675b5da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:21.682701 32141 consensus_meta.cc:287] T d10a78dcb5c94435a12fb4d87b63b1a8 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.634s user 0.006s sys 0.000s
I20260809 14:30:21.683339 32298 tablet_service.cc:1514] Processing DeleteTablet for tablet 200c48c11cb042169b307bc1902be82e with delete_type TABLET_DATA_DELETED (replaced by tablet 64ce0263e6e7484dac027da29836b74e) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:21.685433 32137 consensus_meta.cc:287] T 6812d380b8d143a0bc35c4dc9b2aaeeb P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.507s user 0.008s sys 0.000s
W20260809 14:30:21.698127 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 52421.5us
I20260809 14:30:21.702425 32705 consensus_queue.cc:260] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.707218 31729 master-stress-test.cc:580] Killed test thread 2/10
I20260809 14:30:21.707496 31729 master-stress-test.cc:578] Killing test thread 3/10
I20260809 14:30:21.707798 31729 master-stress-test.cc:580] Killed test thread 3/10
I20260809 14:30:21.708034 31729 master-stress-test.cc:578] Killing test thread 4/10
I20260809 14:30:21.708304 31729 master-stress-test.cc:580] Killed test thread 4/10
I20260809 14:30:21.708701 31729 master-stress-test.cc:578] Killing test thread 5/10
I20260809 14:30:21.709038 31729 master-stress-test.cc:580] Killed test thread 5/10
I20260809 14:30:21.709334 31729 master-stress-test.cc:578] Killing test thread 6/10
I20260809 14:30:21.709962 31729 master-stress-test.cc:580] Killed test thread 6/10
I20260809 14:30:21.710220 31729 master-stress-test.cc:578] Killing test thread 7/10
I20260809 14:30:21.710610 31729 master-stress-test.cc:580] Killed test thread 7/10
I20260809 14:30:21.710848 31729 master-stress-test.cc:578] Killing test thread 8/10
I20260809 14:30:21.652107 1192 leader_election.cc:288] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:21.730509 1205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:30:21.731309 1205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:21.734025 1205 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:30:21.734735 31869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 7 candidate_status { last_received { term: 6 index: 320 } } ignore_live_leader: false dest_uuid: "852dbd0962ae4211923851c7a632768e" is_pre_election: true
I20260809 14:30:21.736706 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d10512ac8b3406098fad1f7da012ab3" candidate_term: 7 candidate_status { last_received { term: 6 index: 320 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
I20260809 14:30:21.739025 711 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d10512ac8b3406098fad1f7da012ab3; no voters: 852dbd0962ae4211923851c7a632768e, ee93a411a1b04d38998018f8c6be59e0
I20260809 14:30:21.740304 1205 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 14:30:21.753403 32691 tablet_bootstrap.cc:492] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:21.753921 32691 ts_tablet_manager.cc:1397] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.266s user 0.013s sys 0.064s
I20260809 14:30:21.770469 32691 raft_consensus.cc:348] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.771412 32691 raft_consensus.cc:374] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:21.771698 32691 raft_consensus.cc:729] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:21.772643 1194 raft_consensus.cc:482] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.772534 32691 consensus_queue.cc:260] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.773308 1194 raft_consensus.cc:504] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.777592 32691 ts_tablet_manager.cc:1428] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260809 14:30:21.788686 1194 leader_election.cc:288] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:21.807336 32146 consensus_meta.cc:287] T f139a623619842418f7cf3db3675b5da P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.594s user 0.004s sys 0.002s
I20260809 14:30:21.809203 32691 tablet_bootstrap.cc:492] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:21.812377 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:21.814013 32162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4af0fdb719405fb8fefb029eb9c71d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:21.819631 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "301b53eeb3be41a2b8ff2598d1b531d7" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:21.821197 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:21.822104 32164 raft_consensus.cc:2454] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:21.823173 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:21.823724 32169 raft_consensus.cc:2454] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:21.826431 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad186f0ccf4c15a4f0b16bfffa945c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:21.827134 32510 raft_consensus.cc:2454] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 0.
I20260809 14:30:21.832852 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:21.833576 32508 raft_consensus.cc:2379] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3eb0bc287b12421d89abf97ca66e4d9b in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
W20260809 14:30:21.707607 32133 consensus_meta.cc:287] T 41960774cfb647e4a7c4534e0afd332e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.529s user 0.009s sys 0.000s
I20260809 14:30:21.844324 996 ts_tablet_manager.cc:1918] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:21.862608 996 log.cc:1196] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/301b53eeb3be41a2b8ff2598d1b531d7
I20260809 14:30:21.865033 996 ts_tablet_manager.cc:1939] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:21.862309 32691 tablet_bootstrap.cc:654] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:21.874944 1207 tablet.cc:1722] T 8f35f6dc9a33438db135d9978caf3b14 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:21.877569 31943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Sending DeleteTablet for 3 replicas of tablet f43385579b29417289f3bdacf8fce278
I20260809 14:30:21.882093 31729 master-stress-test.cc:580] Killed test thread 8/10
I20260809 14:30:21.882346 31729 master-stress-test.cc:578] Killing test thread 9/10
I20260809 14:30:21.884696 32693 tablet_bootstrap.cc:654] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:21.637831 32305 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ce3271ee8684662a9d3cd56157e572a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:47010
W20260809 14:30:21.890810 32134 consensus_meta.cc:287] T ba12d9ba8c0e47879713159348ba1126 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent flushing consensus metadata: real 0.753s user 0.006s sys 0.001s
I20260809 14:30:21.891758 32134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 322) took 1064 ms. Trace:
I20260809 14:30:21.596374 1156 raft_consensus.cc:504] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.893210 32167 raft_consensus.cc:2454] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:21.894132 1156 leader_election.cc:288] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:21.895579 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:21.897404 1020 ts_tablet_manager.cc:1918] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:21.897817 1020 log.cc:1196] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/2f171260cb764ac284aef85e4219767e
I20260809 14:30:21.896986 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:21.898748 1020 ts_tablet_manager.cc:1939] T 2f171260cb764ac284aef85e4219767e P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:21.899001 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:21.899575 32164 raft_consensus.cc:2454] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
W20260809 14:30:21.911592 32439 leader_election.cc:341] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:21.913226 32298 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af4e169f518447c8d29c630dca93ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:21.914129 32439 leader_election.cc:302] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:21.915294 1196 raft_consensus.cc:2781] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:30:21.915194 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4af0fdb719405fb8fefb029eb9c71d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:21.915599 1196 raft_consensus.cc:482] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.915918 1196 raft_consensus.cc:3037] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:21.916728 32439 leader_election.cc:302] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:21.916951 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "301b53eeb3be41a2b8ff2598d1b531d7" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:21.918370 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:21.919901 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:21.920689 1194 raft_consensus.cc:2781] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:30:21.920994 1194 raft_consensus.cc:482] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.921288 1194 raft_consensus.cc:3037] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:21.924846 32439 leader_election.cc:302] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:21.925705 1192 raft_consensus.cc:2781] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:30:21.925995 1192 raft_consensus.cc:482] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:21.926292 1192 raft_consensus.cc:3037] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Advancing to term 2
W20260809 14:30:21.932803 32439 leader_election.cc:341] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:21.934648 32439 leader_election.cc:302] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:21.943806 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:21.944598 32169 raft_consensus.cc:2454] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 0.
I20260809 14:30:21.946064 1192 raft_consensus.cc:504] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:21.952819 1192 leader_election.cc:288] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:21.954013 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:21.955122 32469 tablet_service.cc:1467] Processing CreateTablet for tablet ba12d9ba8c0e47879713159348ba1126 (DEFAULT_TABLE table=default.Table_fe15193782d744c19d096d034729510a [id=05974613d5a54b60b00f6f292ec483e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.955801 32480 tablet_service.cc:1467] Processing CreateTablet for tablet b526764f8d2b43d6942167ffa5d34721 (DEFAULT_TABLE table=default.Table_f01ac5f2b8b54a57a444f006d814b132 [id=9ffc3631d1aa48a88e6aebbf24d9217b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.955899 32141 tablet_service.cc:1514] Processing DeleteTablet for tablet f43385579b29417289f3bdacf8fce278 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:21 UTC) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:21.892069 32134 rpcz_store.cc:276] 0809 14:30:20.827554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.827689 (+ 135us) service_pool.cc:227] Handling call
0809 14:30:20.960561 (+132872us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:20.960659 (+ 98us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:21.137706 (+177047us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.891246 (+753540us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:21.891684 (+ 438us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:21.957031 32479 tablet_service.cc:1467] Processing CreateTablet for tablet 36350e37e5834491a352417543f7564c (DEFAULT_TABLE table=default.Table_0227da888bb24888976eae18fce543e9 [id=bb9c3a0b52f0419db0b22211334a0aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.955819 32476 tablet_service.cc:1467] Processing CreateTablet for tablet d10a78dcb5c94435a12fb4d87b63b1a8 (DEFAULT_TABLE table=default.Table_b27d4b6a8d4d47388f293a343c5d11ed [id=8f6e37550cfa41609d2add1f73f3790e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.958194 32482 tablet_service.cc:1467] Processing CreateTablet for tablet f9be01cb6d944a04992da23002688d91 (DEFAULT_TABLE table=default.Table_153b78d9a7934c7ca382c10459415af0 [id=6a339b319fcc44de9e1a5feae512ba05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.958503 32470 tablet_service.cc:1467] Processing CreateTablet for tablet 93514283ff8e4fad8dae1b76d8efd4c2 (DEFAULT_TABLE table=default.Table_82b4d1f100df47fd8c1c0fed677e4a5b [id=b26f9932b01e4616a9a4f5c96c77385f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260809 14:30:21.960335 31933 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a1bfe0fa1bb246ffb434b7aa9652935f (table default.Table_229f88bff6c54d5186d3aa8190d0ef9e [id=7b60c14a5a764cb9abae551857fb55a1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a1bfe0fa1bb246ffb434b7aa9652935f consensus state has no leader
I20260809 14:30:21.960259 32477 tablet_service.cc:1467] Processing CreateTablet for tablet f8ecf7edec2b48dcbaea4f1413b44f95 (DEFAULT_TABLE table=default.Table_558c3b629f83480ea933b6c8d7ee9d1d [id=7c797610f85a4da59bc2c26738f3f779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.962294 32487 tablet_service.cc:1467] Processing CreateTablet for tablet f6682dd422794fdd85ed33002db448d5 (DEFAULT_TABLE table=default.Table_348f7a78a51f40369f9fbd23580ff6b1 [id=667197f2e47b4910b1d8b1d3dd7f955d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:21.968031 32126 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:41528 (request call id 56) took 1882 ms. Trace:
I20260809 14:30:21.968425 32126 rpcz_store.cc:276] 0809 14:30:20.085873 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.086007 (+ 134us) service_pool.cc:227] Handling call
0809 14:30:21.158775 (+1072768us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400111a54. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181471 00007fb6c71f6268 00007fb6c81e26cd 00007fb6c6a8ae04 00007fb6c6a8ad92 00007fb6c6a8acc1 00007fb6c6a8ac3c 00007fb6c6a89ef2 00007fb6c4fab94c 00007fb6c4faad86 00007fb6c4faa72f
0809 14:30:21.967962 (+809187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":727904640}
I20260809 14:30:21.969800 32137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 314) took 1662 ms. Trace:
I20260809 14:30:21.970057 32146 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 318) took 1306 ms. Trace:
I20260809 14:30:21.970098 32137 rpcz_store.cc:276] 0809 14:30:20.307382 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.307695 (+ 313us) service_pool.cc:227] Handling call
0809 14:30:20.497279 (+189584us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:20.497381 (+ 102us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:21.177890 (+680509us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.708336 (+530446us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:21.969746 (+261410us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":136751}
I20260809 14:30:21.970366 32146 rpcz_store.cc:276] 0809 14:30:20.664007 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.664138 (+ 131us) service_pool.cc:227] Handling call
0809 14:30:20.925181 (+261043us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:20.925301 (+ 120us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:21.212599 (+287298us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.969638 (+757039us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:21.969997 (+ 359us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":161730}
I20260809 14:30:21.971740 32133 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41544 (request call id 313) took 1708 ms. Trace:
I20260809 14:30:21.972179 32133 rpcz_store.cc:276] 0809 14:30:20.263547 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.263866 (+ 319us) service_pool.cc:227] Handling call
0809 14:30:20.300554 (+ 36688us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:20.300655 (+ 101us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:21.177898 (+877243us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.971211 (+793313us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:21.971665 (+ 454us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":133248}
W20260809 14:30:21.972884 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 144745us
W20260809 14:30:22.000583 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 65993.2us
I20260809 14:30:22.001773 1156 raft_consensus.cc:482] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:22.002313 1156 raft_consensus.cc:504] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:22.003897 1202 raft_consensus.cc:482] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:22.004290 1202 raft_consensus.cc:504] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.006258 1202 leader_election.cc:288] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:22.013475 1156 leader_election.cc:288] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:22.019212 32475 tablet_service.cc:1467] Processing CreateTablet for tablet a6d2c214e3e74a2db41f28f84ea4a65c (DEFAULT_TABLE table=default.Table_cad4e455f76b418f90ffde8a67a9566f [id=96e505caf4234af6942276b524209971]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.020150 1193 raft_consensus.cc:2781] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:30:22.020547 1193 raft_consensus.cc:482] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:22.020711 1202 raft_consensus.cc:482] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:22.020816 32473 tablet_service.cc:1514] Processing DeleteTablet for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:21 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:22.020890 1193 raft_consensus.cc:3037] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:22.021127 1202 raft_consensus.cc:504] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.023116 1202 leader_election.cc:288] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:22.023490 1202 raft_consensus.cc:482] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:22.024881 32474 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b196e429514cbc803e9a99a986d4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:22.025620 1197 raft_consensus.cc:482] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:22.026051 1197 raft_consensus.cc:504] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.028230 1197 leader_election.cc:288] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:22.028780 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet e74db9caa7f94059b1bba4a082fe57ef (table default.Table_18bbb19abeca4f54a421b38edeba896c [id=7b5a3b18eb3f4379aaa766e476afb21d]) successfully deleted
I20260809 14:30:22.032748 32474 tablet_service.cc:1514] Processing DeleteTablet for tablet f43385579b29417289f3bdacf8fce278 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:21 UTC) from {username='slave'} at 127.0.0.1:44626
I20260809 14:30:22.034380 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad186f0ccf4c15a4f0b16bfffa945c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:22.034699 1196 raft_consensus.cc:504] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.023909 1202 raft_consensus.cc:504] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.037725 1202 leader_election.cc:288] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:22.042775 32469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba12d9ba8c0e47879713159348ba1126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.044884 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b526764f8d2b43d6942167ffa5d34721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.048939 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36350e37e5834491a352417543f7564c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.050284 32439 leader_election.cc:302] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:22.050976 1197 raft_consensus.cc:2781] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 14:30:22.051293 1197 raft_consensus.cc:482] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:22.051609 1197 raft_consensus.cc:3037] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:22.057518 32476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10a78dcb5c94435a12fb4d87b63b1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.059273 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:21.939828 1194 raft_consensus.cc:504] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.061928 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9be01cb6d944a04992da23002688d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.063099 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ecf7edec2b48dcbaea4f1413b44f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.065080 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6682dd422794fdd85ed33002db448d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.065440 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:22.066615 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:22.067371 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:22.069949 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 270) took 3975 ms. Trace:
I20260809 14:30:22.070291 996 rpcz_store.cc:276] 0809 14:30:18.094241 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.094452 (+ 211us) service_pool.cc:227] Handling call
0809 14:30:21.490473 (+3396021us) ts_tablet_manager.cc:1145] Deleting tablet 301b53eeb3be41a2b8ff2598d1b531d7
0809 14:30:21.490632 (+ 159us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:21.788077 (+297445us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.844081 (+ 56004us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.987996 (+143915us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54000f3b28. stack: 00007f555ab48033 00007f555ab47f30 00007f555a6c9534 00000000004db2b4 00000000004db249 00007f5561e64361 00007f5561eebc1c 00007f5562afae0b 00007f5562b2ae10 00007f5562b11e0d 00007f5562b11db9 00007f5562b11d49 00007f5562b11d11 00007f5562b10f3d 0000000000580784 00000000005805b9
0809 14:30:22.069893 (+ 81897us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":133584,"spinlock_wait_cycles":260611456,"tablet-delete.queue_time_us":3335280}
I20260809 14:30:22.071853 996 tablet_replica.cc:330] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:22.072839 32470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93514283ff8e4fad8dae1b76d8efd4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.073549 32475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d2c214e3e74a2db41f28f84ea4a65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.036442 1196 leader_election.cc:288] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:22.079684 1194 leader_election.cc:288] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:22.081724 996 raft_consensus.cc:2229] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:22.082171 996 raft_consensus.cc:2258] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
W20260809 14:30:22.085383 32095 leader_election.cc:341] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 94c4cbdda4ed48a8b34728d231497d02
I20260809 14:30:22.087121 32163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cb7842617a4779bf110e4ea254e011" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:22.088382 996 ts_tablet_manager.cc:1905] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:22.088825 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:22.089663 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:22.091643 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 301b53eeb3be41a2b8ff2598d1b531d7 (table default.Table_9d422f6e103b4228825e2f4398b4e1e7 [id=89f12138e83d4889b2679f8b83243f1d]) successfully deleted
I20260809 14:30:22.048389 1193 raft_consensus.cc:504] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.093391 32515 raft_consensus.cc:2379] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 2: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
I20260809 14:30:22.094662 1193 leader_election.cc:288] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:22.101028 32301 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 295) took 1167 ms. Trace:
I20260809 14:30:22.060218 31945 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet faad36657cd24202abc0573c9e3aaa90 (table default.Table_6680c0954c634f329cc9a572853054d7 [id=77393de1074146b5ace396d3827d2003]). Expected version 1 got 0
I20260809 14:30:22.106087 31945 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet b393017627934fa1b740d8ba8ac62d51 (table default.Table_bb45fcc479b0477f872971659421b3ac [id=7043672a284145c7a5c9a5d80c322ef8]). Expected version 2 got 0
I20260809 14:30:22.111938 32135 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c4cbdda4ed48a8b34728d231497d02 with delete_type TABLET_DATA_DELETED (replaced by tablet b393017627934fa1b740d8ba8ac62d51) from {username='slave'} at 127.0.0.1:41544
I20260809 14:30:22.113641 1197 raft_consensus.cc:504] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.116120 32691 tablet_bootstrap.cc:492] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:22.116561 32691 ts_tablet_manager.cc:1397] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.308s user 0.014s sys 0.008s
I20260809 14:30:22.101415 32301 rpcz_store.cc:276] 0809 14:30:20.933226 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.016798 (+ 83572us) service_pool.cc:227] Handling call
0809 14:30:21.018364 (+ 1566us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:21.018440 (+ 76us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:21.902282 (+883842us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.059934 (+157652us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:22.100968 (+ 41034us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:22.127527 32691 raft_consensus.cc:348] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.128183 32691 raft_consensus.cc:374] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:22.128414 32691 raft_consensus.cc:729] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:22.129211 32691 consensus_queue.cc:260] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.131672 32691 ts_tablet_manager.cc:1428] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260809 14:30:22.132609 32691 tablet_bootstrap.cc:492] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:22.138772 31945 master_service.cc:946] ReplaceTablet: received request to replace tablet 47089009baac430f9bcecec5dbe87687 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:22.140255 32705 ts_tablet_manager.cc:1428] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.995s user 0.006s sys 0.006s
W20260809 14:30:22.140729 32705 ts_tablet_manager.cc:1460] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Tablet startup took 995ms
W20260809 14:30:22.141055 32705 ts_tablet_manager.cc:1462] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Trace:
0809 14:30:01.551795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:01.552494 (+ 699us) service_pool.cc:227] Handling call
0809 14:30:01.554470 (+ 1976us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:01.554600 (+ 130us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:01.596375 (+ 41775us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:02.307218 (+710843us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:02.307988 (+ 770us) inbound_call.cc:173] Queueing success response
0809 14:30:20.505853 (+18197865us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 14:30:21.145095 (+639242us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 14:30:21.146676 (+ 1581us) tablet_replica.cc:259] Starting instrumentation
0809 14:30:21.147227 (+ 551us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":302535,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":6177,"tablet-open.queue_time_us":18198041,"thread_start_us":36129,"threads_started":1}
I20260809 14:30:22.140101 1197 leader_election.cc:288] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:22.168411 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 117153us
I20260809 14:30:22.181581 1197 raft_consensus.cc:482] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:22.182185 1197 raft_consensus.cc:504] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.184113 1197 leader_election.cc:288] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:22.192287 32439 leader_election.cc:341] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.194382 32396 heartbeater.cc:464] Registering TS with master...
I20260809 14:30:22.195382 32691 tablet_bootstrap.cc:654] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:22.196082 1168 consensus_queue.cc:1035] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260809 14:30:22.201929 32305 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee59b2efd4f4c6c9c8f58fee02c4fd7 (DEFAULT_TABLE table=default.Table_a3796b4fbd05463eb6193485302bcd5e [id=a9b5a8f50bb440be853d4e6dca619354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.203071 32305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee59b2efd4f4c6c9c8f58fee02c4fd7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:22.207159 32300 consensus_meta.cc:287] T 59134b4266764866aca8030f9240e3e7 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.765s user 0.007s sys 0.000s
I20260809 14:30:22.208112 32300 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 291) took 1343 ms. Trace:
I20260809 14:30:22.210824 32298 tablet_service.cc:1467] Processing CreateTablet for tablet f9be01cb6d944a04992da23002688d91 (DEFAULT_TABLE table=default.Table_153b78d9a7934c7ca382c10459415af0 [id=6a339b319fcc44de9e1a5feae512ba05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.214587 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9be01cb6d944a04992da23002688d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.211462 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 93514283ff8e4fad8dae1b76d8efd4c2 (DEFAULT_TABLE table=default.Table_82b4d1f100df47fd8c1c0fed677e4a5b [id=b26f9932b01e4616a9a4f5c96c77385f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.219153 32705 tablet_bootstrap.cc:492] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:22.212904 32302 tablet_service.cc:1467] Processing CreateTablet for tablet f8ecf7edec2b48dcbaea4f1413b44f95 (DEFAULT_TABLE table=default.Table_558c3b629f83480ea933b6c8d7ee9d1d [id=7c797610f85a4da59bc2c26738f3f779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.224078 32306 tablet_service.cc:1467] Processing CreateTablet for tablet f6682dd422794fdd85ed33002db448d5 (DEFAULT_TABLE table=default.Table_348f7a78a51f40369f9fbd23580ff6b1 [id=667197f2e47b4910b1d8b1d3dd7f955d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.220865 32297 tablet_service.cc:1467] Processing CreateTablet for tablet d10a78dcb5c94435a12fb4d87b63b1a8 (DEFAULT_TABLE table=default.Table_b27d4b6a8d4d47388f293a343c5d11ed [id=8f6e37550cfa41609d2add1f73f3790e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.221637 32310 tablet_service.cc:1467] Processing CreateTablet for tablet ba12d9ba8c0e47879713159348ba1126 (DEFAULT_TABLE table=default.Table_fe15193782d744c19d096d034729510a [id=05974613d5a54b60b00f6f292ec483e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.223033 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:22.222700 32294 tablet_service.cc:1467] Processing CreateTablet for tablet a6d2c214e3e74a2db41f28f84ea4a65c (DEFAULT_TABLE table=default.Table_cad4e455f76b418f90ffde8a67a9566f [id=96e505caf4234af6942276b524209971]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.243228 32306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6682dd422794fdd85ed33002db448d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.245321 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faad36657cd24202abc0573c9e3aaa90" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:22.245790 32439 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 51871.7us
I20260809 14:30:22.167830 1004 ts_tablet_manager.cc:1918] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:22.176734 32164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:22.247972 32164 raft_consensus.cc:3037] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Advancing to term 2
W20260809 14:30:22.250032 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 144758us
I20260809 14:30:22.177985 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:22.252727 32165 raft_consensus.cc:3037] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:22.179108 32163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:22.256727 32163 raft_consensus.cc:3037] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 2
I20260809 14:30:22.257604 32301 tablet_service.cc:1467] Processing CreateTablet for tablet 36350e37e5834491a352417543f7564c (DEFAULT_TABLE table=default.Table_0227da888bb24888976eae18fce543e9 [id=bb9c3a0b52f0419db0b22211334a0aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.267926 32163 raft_consensus.cc:2454] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 2.
I20260809 14:30:22.271486 32164 raft_consensus.cc:2454] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 2.
I20260809 14:30:22.273223 32165 raft_consensus.cc:2454] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 2.
I20260809 14:30:22.274044 1004 log.cc:1196] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/2dc8d0cad33b4778af9fe9247743c38f
I20260809 14:30:22.285404 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:22.286527 32302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ecf7edec2b48dcbaea4f1413b44f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.294425 32691 tablet_bootstrap.cc:492] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:22.294860 32691 ts_tablet_manager.cc:1397] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.162s user 0.008s sys 0.003s
W20260809 14:30:22.296944 32439 leader_election.cc:341] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.297555 32439 leader_election.cc:302] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:22.299875 1214 raft_consensus.cc:2781] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 14:30:22.300297 1214 raft_consensus.cc:686] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:22.301066 1214 consensus_queue.cc:237] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.301892 32691 raft_consensus.cc:348] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.302580 32691 raft_consensus.cc:374] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:22.302829 32691 raft_consensus.cc:729] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:22.303416 32691 consensus_queue.cc:260] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.307593 32439 leader_election.cc:302] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:22.308334 32439 leader_election.cc:302] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb; no voters:
I20260809 14:30:22.310667 1196 raft_consensus.cc:2781] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 14:30:22.311105 1196 raft_consensus.cc:686] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:22.311808 1196 consensus_queue.cc:237] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.312633 1197 raft_consensus.cc:2781] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 14:30:22.313109 1197 raft_consensus.cc:686] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:22.314333 32309 tablet_service.cc:1514] Processing DeleteTablet for tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:21 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:22.314060 1197 consensus_queue.cc:237] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.317548 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:22.327605 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:22.316358 1200 log.cc:924] Time spent T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260809 14:30:22.330595 32294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d2c214e3e74a2db41f28f84ea4a65c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:22.336871 1179 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.925s user 0.005s sys 0.014s
I20260809 14:30:22.341827 32691 ts_tablet_manager.cc:1428] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.047s user 0.009s sys 0.000s
I20260809 14:30:22.342895 32691 tablet_bootstrap.cc:492] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:22.354127 31943 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet b393017627934fa1b740d8ba8ac62d51 (table default.Table_bb45fcc479b0477f872971659421b3ac [id=7043672a284145c7a5c9a5d80c322ef8]). Expected version 2 got 0
I20260809 14:30:22.355809 1197 raft_consensus.cc:482] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:22.356609 1197 raft_consensus.cc:504] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.361004 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0a36a6ddc64557b31d0dfef8b5b457" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:22.362995 1197 leader_election.cc:288] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:22.370754 32439 leader_election.cc:341] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.382169 1197 raft_consensus.cc:482] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:22.382695 1197 raft_consensus.cc:504] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.386035 1197 leader_election.cc:288] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:22.348835 32295 consensus_meta.cc:287] T 060e3ed3be3140559a59f7204ce0a2b6 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.849s user 0.007s sys 0.018s
I20260809 14:30:22.349221 1020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 247) took 6361 ms. Trace:
I20260809 14:30:22.351649 32326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cb7842617a4779bf110e4ea254e011" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:22.395128 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 29fdaef8a95e4aa0b23018d384442d16 (table default.Table_3c5c49afb5814baf8e9f3f33f5beac33 [id=0c17902222b34f16aab8173dbd666e86]) successfully deleted
I20260809 14:30:22.398252 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b393017627934fa1b740d8ba8ac62d51" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:22.400353 32439 leader_election.cc:341] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.406594 32691 tablet_bootstrap.cc:654] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:22.352918 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:22.208425 32300 rpcz_store.cc:276] 0809 14:30:20.864698 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.865436 (+ 738us) service_pool.cc:227] Handling call
0809 14:30:20.866761 (+ 1325us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:20.866863 (+ 102us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:21.434088 (+567225us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.207635 (+773547us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:22.208056 (+ 421us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:22.366009 32309 tablet_service.cc:1514] Processing DeleteTablet for tablet f43385579b29417289f3bdacf8fce278 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:21 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:22.370720 32299 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b196e429514cbc803e9a99a986d4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 14:30:20 UTC) from {username='slave'} at 127.0.0.1:47010
I20260809 14:30:22.374135 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:22.375044 32325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:22.377259 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:22.378710 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
W20260809 14:30:22.387535 1186 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s user 0.015s sys 0.018s
W20260809 14:30:22.416384 1186 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.963s user 0.015s sys 0.018s
I20260809 14:30:22.419494 1227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:30:22.420079 1227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:22.422505 1227 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:30:22.424616 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 7 candidate_status { last_received { term: 6 index: 475 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0" is_pre_election: true
W20260809 14:30:22.427716 1179 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.236s user 0.005s sys 0.014s
W20260809 14:30:22.429178 1207 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.553s user 0.010s sys 0.020s
I20260809 14:30:22.389580 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93514283ff8e4fad8dae1b76d8efd4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.392904 1020 rpcz_store.cc:276] 0809 14:30:15.987523 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.988177 (+ 654us) service_pool.cc:227] Handling call
0809 14:30:19.652838 (+3664661us) ts_tablet_manager.cc:1145] Deleting tablet 2f171260cb764ac284aef85e4219767e
0809 14:30:19.652885 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:21.135526 (+1482641us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:21.897179 (+761653us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.349147 (+451968us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3664127}
I20260809 14:30:22.412472 32295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 302) took 1346 ms. Trace:
I20260809 14:30:22.437597 32295 rpcz_store.cc:276] 0809 14:30:21.065614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.223944 (+158330us) service_pool.cc:227] Handling call
0809 14:30:21.394420 (+170476us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:21.394548 (+ 128us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:21.499475 (+104927us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.411646 (+912171us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:22.412344 (+ 698us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:22.448477 32310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba12d9ba8c0e47879713159348ba1126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.449927 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10a78dcb5c94435a12fb4d87b63b1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.451421 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36350e37e5834491a352417543f7564c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 14:30:22.452692 32303 consensus_meta.cc:287] T 11f5459874e841e2b39a0e2c5f4d2f87 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent flushing consensus metadata: real 0.562s user 0.009s sys 0.000s
I20260809 14:30:22.469957 32307 tablet_service.cc:1467] Processing CreateTablet for tablet b526764f8d2b43d6942167ffa5d34721 (DEFAULT_TABLE table=default.Table_f01ac5f2b8b54a57a444f006d814b132 [id=9ffc3631d1aa48a88e6aebbf24d9217b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 14:30:22.487910 32691 tablet_bootstrap.cc:492] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:22.488323 32691 ts_tablet_manager.cc:1397] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.146s user 0.008s sys 0.007s
W20260809 14:30:22.493346 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 242771us
I20260809 14:30:22.512253 1004 ts_tablet_manager.cc:1939] T 2dc8d0cad33b4778af9fe9247743c38f P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:22.513564 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 272) took 4664 ms. Trace:
I20260809 14:30:22.513960 1004 rpcz_store.cc:276] 0809 14:30:17.849258 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.859104 (+ 9846us) service_pool.cc:227] Handling call
0809 14:30:21.003891 (+3144787us) ts_tablet_manager.cc:1145] Deleting tablet 2dc8d0cad33b4778af9fe9247743c38f
0809 14:30:21.003958 (+ 67us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:21.995543 (+991585us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.167553 (+172010us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.513492 (+345939us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3098302}
I20260809 14:30:22.515794 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 2dc8d0cad33b4778af9fe9247743c38f (table default.Table_02301a82edfd45bd9511edf923e1028d [id=c1935d3bc49c405abe4d006932820224]) successfully deleted
I20260809 14:30:22.517218 1004 tablet_replica.cc:330] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:22.518420 1004 raft_consensus.cc:2229] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:22.519104 1004 raft_consensus.cc:2258] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:22.521499 1004 ts_tablet_manager.cc:1905] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:22.512282 32705 tablet_bootstrap.cc:654] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:22.530707 32691 raft_consensus.cc:348] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.531445 32691 raft_consensus.cc:374] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:22.531697 32691 raft_consensus.cc:729] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:22.532291 32691 consensus_queue.cc:260] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.535084 32304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 303) took 1248 ms. Trace:
I20260809 14:30:22.535421 32304 rpcz_store.cc:276] 0809 14:30:21.287187 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.289602 (+ 2415us) service_pool.cc:227] Handling call
0809 14:30:21.608936 (+319334us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:21.631780 (+ 22844us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:22.008954 (+377174us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.473742 (+464788us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:22.535026 (+ 61284us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:22.536706 32296 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 304) took 1240 ms. Trace:
I20260809 14:30:22.537055 32296 rpcz_store.cc:276] 0809 14:30:21.296506 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.360884 (+ 64378us) service_pool.cc:227] Handling call
0809 14:30:21.579401 (+218517us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:21.579848 (+ 447us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:22.102426 (+522578us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.487923 (+385497us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:22.536642 (+ 48719us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:22.538393 32307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b526764f8d2b43d6942167ffa5d34721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 14:30:22.568078 32691 ts_tablet_manager.cc:1428] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.079s user 0.007s sys 0.000s
I20260809 14:30:22.571710 31938 catalog_manager.cc:5495] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 1 to 2, leader changed from 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66) to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 2 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } }
W20260809 14:30:22.585155 1224 log.cc:924] Time spent T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a: Append to log took a long time: real 0.068s user 0.000s sys 0.001s
I20260809 14:30:22.592705 32691 tablet_bootstrap.cc:492] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:22.604050 31938 catalog_manager.cc:5587] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797) does not have the latest schema for tablet a051520649c84898ae634f58f3fb42b3 (table default.Table_0e899792451d4491a6b1fe8e30d1aafe [id=ceb18fb1af684b32b996fb84734407c5]). Expected version 1 got 0
I20260809 14:30:22.605161 31938 catalog_manager.cc:5495] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 1 to 2, leader changed from 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66) to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 2 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } }
W20260809 14:30:22.484782 32330 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.057s user 0.006s sys 0.020s
I20260809 14:30:22.607093 31938 catalog_manager.cc:5495] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a reported cstate change: term changed from 1 to 2, leader changed from 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66) to 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67). New cstate: current_term: 2 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } }
I20260809 14:30:22.661579 819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 7 candidate_status { last_received { term: 6 index: 475 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:30:22.662513 819 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852dbd0962ae4211923851c7a632768e in term 6.
I20260809 14:30:22.663968 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3; no voters: ee93a411a1b04d38998018f8c6be59e0
I20260809 14:30:22.665030 1227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 14:30:22.665364 1227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Starting leader election (detected failure of leader ee93a411a1b04d38998018f8c6be59e0)
I20260809 14:30:22.665700 1227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 6 FOLLOWER]: Advancing to term 7
I20260809 14:30:22.674819 1227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:22.677946 819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 7 candidate_status { last_received { term: 6 index: 475 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3"
I20260809 14:30:22.678001 31959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 7 candidate_status { last_received { term: 6 index: 475 } } ignore_live_leader: false dest_uuid: "ee93a411a1b04d38998018f8c6be59e0"
I20260809 14:30:22.678480 819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 14:30:22.679692 1227 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 7 election: Requested vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:30:22.682031 32124 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:41528 (request call id 57) took 1934 ms. Trace:
I20260809 14:30:22.682466 32124 rpcz_store.cc:276] 0809 14:30:20.747195 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.747345 (+ 150us) service_pool.cc:227] Handling call
0809 14:30:21.208315 (+460970us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b5400017554. stack: 00007fb6c0275033 00007fb6c0274f30 00007fb6bfdf6534 00000000004db2b4 00000000004db249 00007fb6c7181471 00007fb6c71f6268 00007fb6c81e26cd 00007fb6c6a8ae04 00007fb6c6a8ad92 00007fb6c6a8acc1 00007fb6c6a8ac3c 00007fb6c6a89ef2 00007fb6c4fab94c 00007fb6c4faad86 00007fb6c4faa72f
0809 14:30:22.681958 (+1473643us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":56027008}
I20260809 14:30:22.686743 819 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852dbd0962ae4211923851c7a632768e in term 7.
I20260809 14:30:22.687821 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 852dbd0962ae4211923851c7a632768e, 9d10512ac8b3406098fad1f7da012ab3; no voters: ee93a411a1b04d38998018f8c6be59e0
I20260809 14:30:22.690806 32443 leader_election.cc:302] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:22.691607 1196 raft_consensus.cc:2726] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 14:30:22.713896 1227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 7 FOLLOWER]: Leader election won for term 7
I20260809 14:30:22.733534 31940 master_service.cc:946] ReplaceTablet: received request to replace tablet 370a8a83f0c840c1b08d92188e5b62d9 from {username='slave'} at 127.0.0.1:49782
I20260809 14:30:22.742619 32443 leader_election.cc:302] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:22.743739 1196 raft_consensus.cc:2781] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:22.744184 1196 raft_consensus.cc:482] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:22.744570 1196 raft_consensus.cc:3037] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:22.751348 1196 raft_consensus.cc:504] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.753544 1196 leader_election.cc:288] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:22.757364 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac7afd531a746cab7618ead98b4c4d9" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
W20260809 14:30:22.758919 32439 leader_election.cc:341] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.763034 1227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 7 LEADER]: Becoming Leader. State: Replica: 852dbd0962ae4211923851c7a632768e, State: Running, Role: LEADER
I20260809 14:30:22.764509 1227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 475, Committed index: 475, Last appended: 6.475, Last appended by leader: 475, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:22.767760 32691 tablet_bootstrap.cc:654] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:22.770673 1004 ts_tablet_manager.cc:1918] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:22.771236 1004 log.cc:1196] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/79bb4dd74afe4ace94eb1519950a83c0
I20260809 14:30:22.772073 1004 ts_tablet_manager.cc:1939] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
W20260809 14:30:22.776469 31735 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 57341.9us
I20260809 14:30:22.787609 32705 tablet_bootstrap.cc:492] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:22.788048 32705 ts_tablet_manager.cc:1397] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.569s user 0.013s sys 0.013s
I20260809 14:30:22.790263 32705 raft_consensus.cc:348] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.790854 32705 raft_consensus.cc:374] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:22.791106 32705 raft_consensus.cc:729] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:22.791666 32705 consensus_queue.cc:260] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.792440 31869 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 7 LEADER]: Rejecting Update request from peer ee93a411a1b04d38998018f8c6be59e0 for earlier term 6. Current term is 7. Ops: [6.476-6.476]
I20260809 14:30:22.807103 1152 consensus_queue.cc:1046] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 }, Status: INVALID_TERM, Last received: 6.475, Next index: 476, Last known committed idx: 475, Time since last communication: 0.001s
I20260809 14:30:22.820673 32135 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d592b1c4520467791ffd9d36e0d97fd with delete_type TABLET_DATA_DELETED (replaced by tablet 23d2b217278c481e8464d00188c70048) from {username='slave'} at 127.0.0.1:41544
W20260809 14:30:22.554705 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 60612.3us
I20260809 14:30:22.859795 32260 leader_election.cc:302] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters:
I20260809 14:30:22.861157 1168 raft_consensus.cc:2781] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 14:30:22.861510 1168 raft_consensus.cc:482] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 14:30:22.861833 1168 raft_consensus.cc:3037] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:22.862334 1237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 852dbd0962ae4211923851c7a632768e. Latest consensus state: current_term: 7 leader_uuid: "852dbd0962ae4211923851c7a632768e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:30:22.863319 1237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: LEADER
I20260809 14:30:22.874300 1242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 14:30:22.615659 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0a36a6ddc64557b31d0dfef8b5b457" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:22.889880 32443 leader_election.cc:341] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.890327 32443 leader_election.cc:302] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:22.616153 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b393017627934fa1b740d8ba8ac62d51" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:22.872118 32330 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.444s user 0.008s sys 0.027s
I20260809 14:30:22.891816 1226 tablet.cc:1722] T 2a689385d063469595fcae82aa979bb1 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260809 14:30:22.892786 32330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.30.252.67:59643 (request call id 183) took 1491 ms. Trace:
I20260809 14:30:22.880393 32693 tablet_bootstrap.cc:492] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:22.894979 32693 ts_tablet_manager.cc:1397] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 1.795s user 0.012s sys 0.022s
W20260809 14:30:22.902637 32443 leader_election.cc:341] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.903131 32443 leader_election.cc:302] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:22.904386 32443 leader_election.cc:341] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.904847 32443 leader_election.cc:302] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:22.906287 1214 raft_consensus.cc:2726] T 7a4af0fdb719405fb8fefb029eb9c71d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:22.895823 1196 raft_consensus.cc:2726] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:22.905488 32443 raft_consensus.cc:2670] T 301b53eeb3be41a2b8ff2598d1b531d7 P 6bed14aff53a41b2bbf5a13ce55dec5a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:22.931262 1196 raft_consensus.cc:482] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:22.931780 1196 raft_consensus.cc:504] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:22.934926 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d81046d929c48a3a6946f20d8fd7bca" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:22.937005 32439 leader_election.cc:341] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:22.937790 1196 leader_election.cc:288] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:22.965838 1152 consensus_queue.cc:1225] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Peer 852dbd0962ae4211923851c7a632768e log is divergent from this leader: its last log entry 7.476 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 475
I20260809 14:30:22.979017 996 ts_tablet_manager.cc:1918] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:22.989748 1239 raft_consensus.cc:3032] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 14:30:22.993273 996 log.cc:1196] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/273ca6176a184a719538a9455afded4d
W20260809 14:30:22.996675 31930 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3d81046d929c48a3a6946f20d8fd7bca (table default.Table_1ce1e7baa3ab4b38a15aa6009f3dd65c [id=f60105c3fd98462b9367f400abca2c15], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3d81046d929c48a3a6946f20d8fd7bca consensus state has no leader
I20260809 14:30:23.000190 996 ts_tablet_manager.cc:1939] T 273ca6176a184a719538a9455afded4d P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:23.002822 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 273) took 4839 ms. Trace:
I20260809 14:30:23.003612 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 274) took 5153 ms. Trace:
I20260809 14:30:22.990156 1239 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee93a411a1b04d38998018f8c6be59e0, State: Running, Role: LEADER
I20260809 14:30:23.010362 1239 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 475, Committed index: 475, Last appended: 6.476, Last appended by leader: 476, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:23.011674 1239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 6 FOLLOWER]: Advancing to term 7
I20260809 14:30:23.045514 1251 tablet.cc:1722] T d9f01d25eac34d4d998dc457be6a5d9b P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:23.004294 1004 rpcz_store.cc:276] 0809 14:30:17.850250 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.871574 (+ 21324us) service_pool.cc:227] Handling call
0809 14:30:22.517059 (+4645485us) ts_tablet_manager.cc:1145] Deleting tablet 79bb4dd74afe4ace94eb1519950a83c0
0809 14:30:22.517100 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:22.616205 (+ 99105us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.770403 (+154198us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.003518 (+233115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4519310}
I20260809 14:30:23.050731 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet 79bb4dd74afe4ace94eb1519950a83c0 (table default.Table_13a67d50c6c746e99d3c0ce47b81f6ad [id=20f2c60561964f50889f993f7a9421f1]) successfully deleted
I20260809 14:30:23.052098 1004 tablet_replica.cc:330] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
W20260809 14:30:23.061085 32099 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.252.66:40891 timed out after 3.892s (SENT)
I20260809 14:30:22.611343 32303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 293) took 1697 ms. Trace:
I20260809 14:30:22.641484 1020 tablet_replica.cc:330] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:23.073426 32303 rpcz_store.cc:276] 0809 14:30:20.865328 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.929681 (+ 64353us) service_pool.cc:227] Handling call
0809 14:30:20.931014 (+ 1333us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:20.931095 (+ 81us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:21.889928 (+958833us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.453302 (+563374us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:22.562740 (+109438us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:23.085770 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_23ccc6f75d894549beac50decbabd886 [id=02a21b34d14e429d954e11d78733643f]
I20260809 14:30:22.617631 32324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faad36657cd24202abc0573c9e3aaa90" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.092975 1233 tablet.cc:1722] T d885baba6d6045e8b1ebcab6513719bb P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:23.096539 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_189adb7a0cf1426aaff4dd77f5991e8f [id=02ea010bf4f346a694169273ea7684ce]
I20260809 14:30:23.096856 1207 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47010 (request call id 265) took 4619 ms. Trace:
I20260809 14:30:23.099999 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_3aa29ab15fd44baa97cb10fe5b9e96cf [id=032899eca6f3408aaf32fa1078ac26fd]
I20260809 14:30:23.102222 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_e6381d3c953a4958ada0a38267050cc0 [id=03607042808746c19f9e8a26eb7fa135]
I20260809 14:30:23.104056 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_45f0fdfd0b17414694813aa5025f9759 [id=0512c169520d416c9797c802d0c4338c]
I20260809 14:30:23.129750 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_fe15193782d744c19d096d034729510a [id=05974613d5a54b60b00f6f292ec483e3]
I20260809 14:30:23.141583 1168 raft_consensus.cc:504] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.143868 1168 leader_election.cc:288] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Requested vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:23.144112 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_60564937e4494c0ca93832cb7817a76c [id=074d8baf9d43478cb631d61a71d58225]
I20260809 14:30:23.149501 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_2c3e9ee14404475bba64491f9d132524 [id=07a8421008714d8eb96d3d29c06c8fc5]
I20260809 14:30:23.097199 1207 rpcz_store.cc:276] 0809 14:30:18.477612 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.477967 (+ 355us) service_pool.cc:227] Handling call
0809 14:30:23.081996 (+4604029us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:18.790114 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:18.790853 (+ 739us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:18.790910 (+ 57us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:18.790993 (+ 83us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:18.791143 (+ 150us) alter_schema_op.cc:126] START. Timestamp: 1786285818791017
0809 14:30:18.791172 (+ 29us) op_driver.cc:331] REPLICATION: starting
0809 14:30:18.792085 (+ 913us) log.cc:842] Serialized 173 byte log entry
0809 14:30:19.317332 (+525247us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b54001999a0. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8398016471 00007f839801f771 00007f839802a220 00007f839802a1c8 00007f839802a148 00007f839802a103 00007f839802938c 00007f839a014206 00007f839a00c808 00007f839806638b 00007f8398068e80
0809 14:30:21.697090 (+2379758us) op_driver.cc:444] REPLICATION: finished
0809 14:30:21.874759 (+177669us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:21.900807 (+ 26048us) log.cc:842] Serialized 5 byte log entry
0809 14:30:22.687560 (+786753us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.871809 (+184249us) log.cc:842] Serialized 14 byte log entry
0809 14:30:22.889234 (+ 17425us) alter_schema_op.cc:71] Released schema lock
0809 14:30:22.894131 (+ 4897us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":177328,"prepare.queue_time_us":311122,"prepare.run_cpu_time_us":6458,"prepare.run_wall_time_us":481212,"raft.queue_time_us":113,"raft.run_cpu_time_us":1345,"raft.run_wall_time_us":1345,"replication_time_us":2905739,"spinlock_wait_cycles":31116544,"thread_start_us":1696084,"threads_started":4,"wal-append.queue_time_us":831566,"wal-append.run_cpu_time_us":5555,"wal-append.run_wall_time_us":1496520}]]}
I20260809 14:30:23.161998 1169 raft_consensus.cc:482] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1f94e0d792c46c9a0b6ab42820da6eb)
I20260809 14:30:23.162420 1169 raft_consensus.cc:504] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.166989 32693 raft_consensus.cc:348] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.167601 32693 raft_consensus.cc:374] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:23.167815 32693 raft_consensus.cc:729] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:23.169016 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_dadc1d59ab5f4b799d8d5d1cf23a3580 [id=0b1913c09d3342d3a9f79d3984530c94]
I20260809 14:30:23.168337 32693 consensus_queue.cc:260] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.164228 1169 leader_election.cc:288] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:23.175843 819 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 7 FOLLOWER]: Refusing update from remote peer 852dbd0962ae4211923851c7a632768e: Log matching property violated. Preceding OpId in replica: term: 6 index: 320. Preceding OpId from leader: term: 7 index: 476. (index mismatch)
I20260809 14:30:23.177765 1227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 476, Last known committed idx: 320, Time since last communication: 0.001s
I20260809 14:30:23.174288 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_5352320f0eb24ca5a85ceb658929954a [id=0cc322f8067843d98204a74e6016db47]
I20260809 14:30:23.186381 1020 raft_consensus.cc:2229] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:22.893136 32330 rpcz_store.cc:276] 0809 14:30:21.401606 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.401723 (+ 117us) service_pool.cc:227] Handling call
0809 14:30:21.402128 (+ 405us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 14:30:21.402374 (+ 246us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 14:30:21.402404 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 14:30:21.426927 (+ 24523us) alter_schema_op.cc:126] START. Timestamp: 1786285819747517
0809 14:30:21.427784 (+ 857us) log.cc:842] Serialized 173 byte log entry
0809 14:30:22.892439 (+1464655us) raft_consensus.cc:1631] Marking committed up to 1
0809 14:30:22.892497 (+ 58us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 14:30:22.892577 (+ 80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 14:30:22.892616 (+ 39us) raft_consensus.cc:1671] finished
0809 14:30:22.892628 (+ 12us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 14:30:22.892734 (+ 106us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:21.428307 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:21.428671 (+ 364us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:21.428700 (+ 29us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:21.428765 (+ 65us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
Metrics: {"thread_start_us":1464345,"threads_started":1,"wal-append.queue_time_us":763543,"child_traces":[["op",{"prepare.queue_time_us":1083,"prepare.run_cpu_time_us":508,"prepare.run_wall_time_us":508}]]}
I20260809 14:30:23.190910 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_c564d1dbe35c421dbd0d9ee2d9b2ece5 [id=0dfa186125334c5b8b5f97b8ffbfaed8]
I20260809 14:30:23.195108 32166 raft_consensus.cc:1261] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Refusing update from remote peer 6bed14aff53a41b2bbf5a13ce55dec5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 14:30:23.200558 996 rpcz_store.cc:276] 0809 14:30:18.163107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.163229 (+ 122us) service_pool.cc:227] Handling call
0809 14:30:22.071665 (+3908436us) ts_tablet_manager.cc:1145] Deleting tablet 273ca6176a184a719538a9455afded4d
0809 14:30:22.071731 (+ 66us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:22.143850 (+ 72119us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.880879 (+737029us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.002726 (+121847us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3907764}
I20260809 14:30:23.201875 996 tablet_replica.cc:330] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:23.202034 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet 273ca6176a184a719538a9455afded4d (table default.Table_43f73cfcef6546fb985efbe413dff035 [id=72bef9dd3c5e43dc9e77990689fb7732]) successfully deleted
I20260809 14:30:23.202642 996 raft_consensus.cc:2229] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:23.201234 1214 consensus_queue.cc:1035] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 14:30:23.203083 996 raft_consensus.cc:2258] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:23.006598 32691 tablet_bootstrap.cc:492] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:23.204737 32691 ts_tablet_manager.cc:1397] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.621s user 0.004s sys 0.008s
I20260809 14:30:23.207633 32691 raft_consensus.cc:348] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.057339 1250 tablet.cc:1722] T d9f01d25eac34d4d998dc457be6a5d9b P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:23.113131 32443 leader_election.cc:302] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:23.212919 1214 raft_consensus.cc:2726] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:30:23.213048 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 100009us
I20260809 14:30:23.192384 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_cf369e8c18034e71a7603e77b8fd94ca [id=0ebd08ac8f7d40fb8aa6d6d0a4a4234e]
I20260809 14:30:23.214710 32443 leader_election.cc:302] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:23.215759 1196 raft_consensus.cc:2726] T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:23.216683 32443 leader_election.cc:302] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:23.217278 1196 raft_consensus.cc:2726] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 14:30:23.218146 32443 leader_election.cc:302] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:23.218757 1196 raft_consensus.cc:2726] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260809 14:30:23.219373 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 158363us
I20260809 14:30:23.226603 32169 raft_consensus.cc:1261] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Refusing update from remote peer 6bed14aff53a41b2bbf5a13ce55dec5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 14:30:23.228216 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_9527df93a9844cf2b0b2cbe5a98d6cae [id=11b91dc887234dc6ba9608566d4547a9]
I20260809 14:30:23.232674 32310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 318) took 1027 ms. Trace:
I20260809 14:30:23.233193 32310 rpcz_store.cc:276] 0809 14:30:22.205371 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.219114 (+ 13743us) service_pool.cc:227] Handling call
0809 14:30:22.231044 (+ 11930us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:22.231114 (+ 70us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:22.740164 (+509050us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.155627 (+415463us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:23.232600 (+ 76973us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:23.236653 31959 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 7 FOLLOWER]: Refusing update from remote peer 852dbd0962ae4211923851c7a632768e: Log matching property violated. Preceding OpId in replica: term: 6 index: 476. Preceding OpId from leader: term: 7 index: 476. (term mismatch)
I20260809 14:30:23.237082 31959 pending_rounds.cc:76] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Aborting all ops after (but not including) 475
I20260809 14:30:23.237299 31959 pending_rounds.cc:97] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0: Aborting uncommitted WRITE_OP operation due to leader change: 6.476
I20260809 14:30:23.240025 1237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 476, Last known committed idx: 475, Time since last communication: 0.000s
E20260809 14:30:23.239583 31938 catalog_manager.cc:5650] Error updating tablets from 6bed14aff53a41b2bbf5a13ce55dec5a: Aborted: Op aborted by new leader. Tablet report was: is_incremental: true updated_tablets { tablet_id: "67afcb47ffc947c5b1d98c96de333116" state: RUNNING consensus_state { current_term: 2 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } } } schema_version: 2 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27066 live_row_count: 0 } } updated_tablets { tablet_id: "8b53f7d398d64701b4504db5c9c440b8" state: RUNNING consensus_state { current_term: 2 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } } } schema_version: 2 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27066 live_row_count: 0 } } updated_tablets { tablet_id: "a051520649c84898ae634f58f3fb42b3" state: RUNNING consensus_state { current_term: 2 leader_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26844 live_row_count: 0 } } updated_tablets { tablet_id: "7856076c1332479a894d2d69d16b19f5" state: RUNNING consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } sequence_number: 59
I20260809 14:30:23.250319 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_76435d57443044e89e81a802eb4853b3 [id=124038878ed44174a413bd6ffa2e836d]
I20260809 14:30:23.250908 1004 raft_consensus.cc:2229] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:23.251637 1004 raft_consensus.cc:2258] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:23.252750 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_a5839d042df748b5b52689e11f581dd6 [id=12ebe7f1d8a149b2a40db412119ef8cb]
I20260809 14:30:23.254232 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_e6905fa41eb64a4c9eeabed102a72d23 [id=1544a31f65074713b012206c3634a8c8]
I20260809 14:30:23.254266 1004 ts_tablet_manager.cc:1905] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:23.255635 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_7077d276f84a485c840fded026cca74a [id=15a8f74e5d2d458abd4779617f33f2ef]
I20260809 14:30:23.264870 1260 consensus_queue.cc:1035] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 14:30:23.208415 32691 raft_consensus.cc:374] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:23.273088 32691 raft_consensus.cc:729] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:23.251048 32705 ts_tablet_manager.cc:1428] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.463s user 0.010s sys 0.001s
I20260809 14:30:23.276260 32705 tablet_bootstrap.cc:492] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:23.282299 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 54685.7us
I20260809 14:30:23.283181 32308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 323) took 1073 ms. Trace:
I20260809 14:30:23.286216 32307 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 317) took 1100 ms. Trace:
I20260809 14:30:23.287359 32301 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 319) took 1078 ms. Trace:
I20260809 14:30:23.283648 32308 rpcz_store.cc:276] 0809 14:30:22.210066 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.210907 (+ 841us) service_pool.cc:227] Handling call
0809 14:30:22.215413 (+ 4506us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:22.215496 (+ 83us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:22.723315 (+507819us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:22.984185 (+260870us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:23.283119 (+298934us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":90366}
I20260809 14:30:23.286610 32307 rpcz_store.cc:276] 0809 14:30:22.185650 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.469288 (+283638us) service_pool.cc:227] Handling call
0809 14:30:22.537923 (+ 68635us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:22.538017 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:22.746847 (+208830us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.236096 (+489249us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:23.286161 (+ 50065us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1385}
I20260809 14:30:23.287715 32301 rpcz_store.cc:276] 0809 14:30:22.208867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.257005 (+ 48138us) service_pool.cc:227] Handling call
0809 14:30:22.258169 (+ 1164us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:22.258247 (+ 78us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:22.762480 (+504233us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.284708 (+522228us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:23.287307 (+ 2599us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2134}
I20260809 14:30:23.293524 32302 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 324) took 1080 ms. Trace:
I20260809 14:30:23.293874 32302 rpcz_store.cc:276] 0809 14:30:22.211774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.212008 (+ 234us) service_pool.cc:227] Handling call
0809 14:30:22.223670 (+ 11662us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:22.223754 (+ 84us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:22.682198 (+458444us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.154970 (+472772us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:23.292170 (+137200us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":46105}
I20260809 14:30:23.294914 1020 raft_consensus.cc:2258] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:23.306721 32294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 326) took 1094 ms. Trace:
I20260809 14:30:23.308281 996 ts_tablet_manager.cc:1905] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:23.307536 32294 rpcz_store.cc:276] 0809 14:30:22.212546 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.212761 (+ 215us) service_pool.cc:227] Handling call
0809 14:30:22.241386 (+ 28625us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:22.242110 (+ 724us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:23.094222 (+852112us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.305885 (+211663us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:23.306637 (+ 752us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:23.310060 32305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47010 (request call id 315) took 1634 ms. Trace:
I20260809 14:30:23.310729 32305 rpcz_store.cc:276] 0809 14:30:21.912581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.996539 (+ 83958us) service_pool.cc:227] Handling call
0809 14:30:22.202619 (+206080us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:22.202704 (+ 85us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:23.098256 (+895552us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.165583 (+ 67327us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:23.309950 (+144367us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":90831}
I20260809 14:30:23.317562 1196 raft_consensus.cc:482] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:23.318442 1196 raft_consensus.cc:504] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.326655 1214 raft_consensus.cc:482] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:23.327299 1214 raft_consensus.cc:504] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.275274 32691 consensus_queue.cc:260] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.356163 1020 ts_tablet_manager.cc:1905] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:23.358608 1001 mvcc.cc:205] Tried to move back new op lower bound from 7316626685451517952 to 7316626605812477952. Current Snapshot: MvccSnapshot[applied={T|T < 7316626685451517952}]
I20260809 14:30:23.365756 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_9e117bba409147cf8e8ec3e5443a321c [id=1995378f5e3e4e18944d14760115b1cc]
I20260809 14:30:23.367802 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_7930652dac8a4ec885cd18e733b1e84e [id=1dd4e785ac184325be3876d8148799f3]
W20260809 14:30:23.378427 1226 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.485s user 0.003s sys 0.013s
I20260809 14:30:23.387486 1196 leader_election.cc:288] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:23.389562 32166 raft_consensus.cc:1261] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Refusing update from remote peer 6bed14aff53a41b2bbf5a13ce55dec5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260809 14:30:23.379611 1233 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.286s user 0.009s sys 0.005s
W20260809 14:30:23.380038 32260 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.516s user 0.010s sys 0.005s
I20260809 14:30:23.394001 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:23.402293 32260 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.538s user 0.011s sys 0.005s
W20260809 14:30:23.405298 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 541426us
I20260809 14:30:23.413909 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_20cad555a34f4f70a19814ffcb820bbe [id=20d2073f354649b3b4bc4d583660c93a]
I20260809 14:30:23.424214 1239 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 852dbd0962ae4211923851c7a632768e. Latest consensus state: current_term: 7 leader_uuid: "852dbd0962ae4211923851c7a632768e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:30:23.425784 1239 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 14:30:23.429561 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f35f6dc9a33438db135d9978caf3b14" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.431211 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_a7b9b0894c4a47afb6848436134e53fc [id=20f5ef70cc1845c9ba5e3a6d3b1a9c1d]
W20260809 14:30:23.434906 32443 leader_election.cc:341] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.435945 32443 leader_election.cc:302] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:23.438619 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_aba0145218af4c9294652147cd9b649a [id=2148e197cec94a68a55c9d20e580b004]
W20260809 14:30:23.449368 32260 leader_election.cc:341] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: e846e795144e48e3b2a1eca8c1b26489
I20260809 14:30:23.450220 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_d7a3df21ac0a4fd2b7cb295edfb62f96 [id=217270d577db418e87628191169460be]
I20260809 14:30:23.450986 1168 consensus_queue.cc:1035] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 14:30:23.463145 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f17af7f81741c68bafa73c2c3d91af" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:23.470299 32443 leader_election.cc:341] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.473984 32443 leader_election.cc:302] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:23.470531 1267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "852dbd0962ae4211923851c7a632768e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:30:23.474965 1267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [sys.catalog]: This master's current role is: LEADER
I20260809 14:30:23.479274 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_6d1220a2972747a7b2050e1714d86647 [id=217a91e1a58b443d93738772edc80b76]
I20260809 14:30:23.482554 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_ba8cab0449354ab3aa7efe707df20ab3 [id=225ab3132a00452ebedc7f1c7167ed9f]
I20260809 14:30:23.500864 32705 tablet_bootstrap.cc:654] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:23.504331 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_a50cf76d58cc49c28888e4e567a71d0d [id=261eb132302e4d74bb18d23eea8ee9d0]
I20260809 14:30:23.511646 1239 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "852dbd0962ae4211923851c7a632768e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:30:23.513792 1239 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 14:30:23.475091 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 105435us
W20260809 14:30:23.518136 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 47868.1us
W20260809 14:30:23.519037 32443 leader_election.cc:341] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.489455 1214 leader_election.cc:288] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:23.476764 1266 raft_consensus.cc:2726] T b393017627934fa1b740d8ba8ac62d51 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:23.519258 32691 ts_tablet_manager.cc:1428] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.314s user 0.010s sys 0.004s
I20260809 14:30:23.519764 1266 raft_consensus.cc:2726] T ea0a36a6ddc64557b31d0dfef8b5b457 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:23.520641 32691 tablet_bootstrap.cc:492] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:23.522177 32166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fa25dbd78f64e5db9e31d433e8f4736" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:23.520900 1266 consensus_queue.cc:1035] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 14:30:23.524806 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7856076c1332479a894d2d69d16b19f5" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:23.524211 32439 leader_election.cc:341] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:23.534109 32439 leader_election.cc:341] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
W20260809 14:30:23.541800 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 70289.8us
I20260809 14:30:23.547363 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_c6a19ad0e6594d42833aa8edc7b517ad [id=2a218160937e463b8cb540104f4fee4b]
I20260809 14:30:23.555272 1250 tablet.cc:1722] T 8f35f6dc9a33438db135d9978caf3b14 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:23.567163 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_9f57196e69e94c7c8e1f34e2b75a48c6 [id=2adef452195e4222ad6c477cbb6e7d70]
I20260809 14:30:23.565325 1251 tablet.cc:1722] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:23.570248 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_fd0e2b9e56354188a3ab1f20bccf533a [id=2baec8ebb5d74a6cb75f9c0eac1728e1]
I20260809 14:30:23.573379 32324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b53f7d398d64701b4504db5c9c440b8" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.575582 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051520649c84898ae634f58f3fb42b3" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.581416 32324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eeed4a20d6413f873585c95bb2858a" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.583681 32324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67afcb47ffc947c5b1d98c96de333116" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.585484 996 ts_tablet_manager.cc:1918] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:23.586018 32324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac7afd531a746cab7618ead98b4c4d9" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b"
I20260809 14:30:23.586833 32324 raft_consensus.cc:3037] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:23.594210 32705 tablet_bootstrap.cc:492] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:23.594686 32705 ts_tablet_manager.cc:1397] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.319s user 0.006s sys 0.007s
I20260809 14:30:23.597739 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_60947825f97f4a7981e3301069cbd4ca [id=2cf0c0e1fb6f40c29bb391b38988e91a]
I20260809 14:30:23.600677 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_24df220422ea4f1ab4d5189b0b5ffc80 [id=2d3c076f6ef54647beba4d1564fb9924]
I20260809 14:30:23.603199 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_423d2129e8fa4a59a8b4a7e853467ffa [id=2d493ef1e83d47da9117ecd5312dacdc]
I20260809 14:30:23.612023 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d81046d929c48a3a6946f20d8fd7bca" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:23.605738 32573 tablet.cc:2369] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 14:30:23.629281 1264 raft_consensus.cc:482] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3eb0bc287b12421d89abf97ca66e4d9b)
I20260809 14:30:23.629887 1264 raft_consensus.cc:504] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } }
I20260809 14:30:23.632220 996 log.cc:1196] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/faf222c49c934d52a2642f20e8575702
I20260809 14:30:23.633014 996 ts_tablet_manager.cc:1939] T faf222c49c934d52a2642f20e8575702 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:23.635501 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 271) took 5480 ms. Trace:
I20260809 14:30:23.636103 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad186f0ccf4c15a4f0b16bfffa945c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb"
I20260809 14:30:23.635869 996 rpcz_store.cc:276] 0809 14:30:18.154930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.155129 (+ 199us) service_pool.cc:227] Handling call
0809 14:30:23.201707 (+5046578us) ts_tablet_manager.cc:1145] Deleting tablet faf222c49c934d52a2642f20e8575702
0809 14:30:23.201763 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:23.494068 (+292305us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.585229 (+ 91161us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.635447 (+ 50218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5037568}
I20260809 14:30:23.637185 31930 catalog_manager.cc:4841] TS 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): tablet faf222c49c934d52a2642f20e8575702 (table default.Table_1f43d253ff624d0181e6801171d88ce4 [id=ca4bf32b107640319ee009cf2f3e4062]) successfully deleted
I20260809 14:30:23.638664 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:23.614977 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.642493 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_089b0a96a71948bd8bc0df4624d303f2 [id=3016ad78c9fd4f8f89874c7da5dcf3c3]
I20260809 14:30:23.644901 1264 leader_election.cc:288] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:23.645821 32514 raft_consensus.cc:1226] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 LEADER]: Rejecting Update request from peer 3eb0bc287b12421d89abf97ca66e4d9b for earlier term 1. Current term is 2. Ops: []
I20260809 14:30:23.650389 799 ts_manager.cc:194] Registered new tserver with Master: 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:23.652089 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_f4a40a5b9dd74b979cde711f5d4d2873 [id=330557f13a9e4189b6492e7bb3327021]
I20260809 14:30:23.654840 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_e6685adbd7a7408f9fc60d67696e3b04 [id=33608e9192854140a2022cb0b0aa92f6]
I20260809 14:30:23.622735 32691 tablet_bootstrap.cc:654] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:23.659164 32163 raft_consensus.cc:1203] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.2] Dedup: 1.2->[]
I20260809 14:30:23.659446 32705 raft_consensus.cc:348] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.660146 32163 raft_consensus.cc:1226] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Rejecting Update request from peer 3eb0bc287b12421d89abf97ca66e4d9b for earlier term 1. Current term is 2. Ops: [1.1-1.2]
I20260809 14:30:23.660323 32705 raft_consensus.cc:374] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:23.660650 32705 raft_consensus.cc:729] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:23.661377 32705 consensus_queue.cc:260] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.662652 32169 raft_consensus.cc:3037] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:23.663865 32705 ts_tablet_manager.cc:1428] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.069s user 0.010s sys 0.001s
I20260809 14:30:23.665031 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:23.669206 32443 leader_election.cc:302] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:23.677009 1004 ts_tablet_manager.cc:1918] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:23.677906 1004 log.cc:1196] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/c468382a28934d50babf7bbc9a834c08
I20260809 14:30:23.682132 1004 ts_tablet_manager.cc:1939] T c468382a28934d50babf7bbc9a834c08 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:23.688004 996 tablet_replica.cc:330] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:23.689213 996 raft_consensus.cc:2229] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:23.693010 32513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad186f0ccf4c15a4f0b16bfffa945c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a"
I20260809 14:30:23.693696 32513 raft_consensus.cc:3037] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:23.696636 32509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28a765e1854486595683c0ac00da24c" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:23.713251 32510 raft_consensus.cc:1203] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 LEADER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 14:30:23.714322 32510 raft_consensus.cc:1226] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 LEADER]: Rejecting Update request from peer 3eb0bc287b12421d89abf97ca66e4d9b for earlier term 1. Current term is 2. Ops: [1.2-1.2]
I20260809 14:30:23.715950 1259 raft_consensus.cc:2726] T faad36657cd24202abc0573c9e3aaa90 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:23.619542 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c4cbdda4ed48a8b34728d231497d02" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.725382 1261 raft_consensus.cc:482] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1f94e0d792c46c9a0b6ab42820da6eb)
I20260809 14:30:23.726498 32325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d885baba6d6045e8b1ebcab6513719bb" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.733213 1261 raft_consensus.cc:504] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.735335 1261 leader_election.cc:288] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:23.739164 996 raft_consensus.cc:2258] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:23.745828 32513 raft_consensus.cc:2454] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0bc287b12421d89abf97ca66e4d9b in term 1.
I20260809 14:30:23.753997 32693 ts_tablet_manager.cc:1428] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.859s user 0.005s sys 0.014s
W20260809 14:30:23.754302 32693 ts_tablet_manager.cc:1460] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b: Tablet startup took 859ms
W20260809 14:30:23.758945 32229 tablet.cc:2369] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 14:30:23.783609 996 ts_tablet_manager.cc:1905] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:23.789129 32325 raft_consensus.cc:3032] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Stepping down as leader of term 1
I20260809 14:30:23.789616 32325 raft_consensus.cc:729] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:23.790746 32325 consensus_queue.cc:260] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.792791 32325 raft_consensus.cc:3037] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:23.796998 1259 raft_consensus.cc:482] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:23.797783 1259 raft_consensus.cc:504] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.801581 1259 leader_election.cc:288] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:23.805152 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8b48c8cde4f11a53eec939a1f8e3d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:23.807498 32439 leader_election.cc:341] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.809625 1168 raft_consensus.cc:482] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1f94e0d792c46c9a0b6ab42820da6eb)
I20260809 14:30:23.810297 1168 raft_consensus.cc:504] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.813585 1168 leader_election.cc:288] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:23.815317 32260 leader_election.cc:341] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.815672 32260 leader_election.cc:302] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:23.816499 1252 raft_consensus.cc:2726] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 14:30:23.817538 32705 tablet_bootstrap.cc:492] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
W20260809 14:30:23.818466 32260 leader_election.cc:341] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Not found: Tablet not found: e846e795144e48e3b2a1eca8c1b26489
I20260809 14:30:23.818817 32260 leader_election.cc:302] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:23.754578 32693 ts_tablet_manager.cc:1462] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b: Trace:
0809 14:30:03.767194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:03.768930 (+ 1736us) service_pool.cc:227] Handling call
0809 14:30:03.777831 (+ 8901us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 14:30:03.778046 (+ 215us) ts_tablet_manager.cc:676] Creating new metadata...
0809 14:30:03.921921 (+143875us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:03.993296 (+ 71375us) tablet_replica.cc:205] Creating consensus instance
0809 14:30:03.994133 (+ 837us) inbound_call.cc:173] Queueing success response
0809 14:30:21.100304 (+17106171us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 14:30:22.895238 (+1794934us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 14:30:23.157986 (+262748us) tablet_replica.cc:259] Starting instrumentation
0809 14:30:23.158431 (+ 445us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":80331,"raft.queue_time_us":142725,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":1303,"spinlock_wait_cycles":1408,"tablet-open.queue_time_us":17106353,"thread_start_us":337069,"threads_started":1}
I20260809 14:30:23.833739 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 278) took 5725 ms. Trace:
I20260809 14:30:23.834230 1004 rpcz_store.cc:276] 0809 14:30:18.108290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.112827 (+ 4537us) service_pool.cc:227] Handling call
0809 14:30:23.051843 (+4939016us) ts_tablet_manager.cc:1145] Deleting tablet c468382a28934d50babf7bbc9a834c08
0809 14:30:23.051905 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:23.537335 (+485430us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.676671 (+139336us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.833660 (+156989us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":56468,"tablet-delete.queue_time_us":4937099}
I20260809 14:30:23.835728 31930 catalog_manager.cc:4841] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): tablet c468382a28934d50babf7bbc9a834c08 (table default.Table_d1d760ad86f94839bf3b5b71dd8809aa [id=44f0605ad9e4418ca64ceca2a4120cf0]) successfully deleted
I20260809 14:30:23.837031 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 285) took 5521 ms. Trace:
I20260809 14:30:23.837352 1004 rpcz_store.cc:276] 0809 14:30:18.315422 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.316538 (+ 1116us) service_pool.cc:227] Handling call
0809 14:30:23.836693 (+5520155us) ts_tablet_manager.cc:1145] Deleting tablet 5ad5bf5ed5cc4c66a4c87fc21209c555
0809 14:30:23.836732 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:23.836982 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5519605}
W20260809 14:30:23.839082 31930 catalog_manager.cc:4821] TS e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): delete failed for tablet 5ad5bf5ed5cc4c66a4c87fc21209c555 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ad5bf5ed5cc4c66a4c87fc21209c555 already in progress: creating tablet
I20260809 14:30:23.839783 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ad14cb3a642e08bace0759bc0b732" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:23.840278 1004 tablet_replica.cc:330] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:23.841882 1004 raft_consensus.cc:2229] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:23.841857 32325 raft_consensus.cc:1261] T 67afcb47ffc947c5b1d98c96de333116 P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Refusing update from remote peer 6bed14aff53a41b2bbf5a13ce55dec5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 14:30:23.842312 1004 pending_rounds.cc:62] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Trying to abort 1 pending ops.
I20260809 14:30:23.842525 1004 pending_rounds.cc:68] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316626724667428864 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "239ad14cb3a642e08bace0759bc0b732" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_e7dbc14f657745d390a15586fda27677" dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" new_extra_config { } }
I20260809 14:30:23.845404 1004 raft_consensus.cc:2258] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:23.847592 1004 ts_tablet_manager.cc:1905] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:23.853309 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_46cf63f7388349388957842c51da95d6 [id=33a64d0e281e4939bef7f46be25aef43]
I20260809 14:30:23.854538 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 2f171260cb764ac284aef85e4219767e (table default.Table_42864987ccd14c05a6909a6e1527e208 [id=032f6264a2c842f1ba6b007331ccfcc6]) successfully deleted
I20260809 14:30:23.855191 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_1c716de0a0fa40f7aa65ec405fceac00 [id=344328f451854f2b8098c8ce1683c55e]
I20260809 14:30:23.858004 32693 tablet_bootstrap.cc:492] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:23.860757 1020 ts_tablet_manager.cc:1918] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260809 14:30:23.888528 32099 leader_election.cc:341] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 79bb4dd74afe4ace94eb1519950a83c0
I20260809 14:30:23.888929 32099 leader_election.cc:302] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:30:23.889456 32099 raft_consensus.cc:2670] T 79bb4dd74afe4ace94eb1519950a83c0 P e1f94e0d792c46c9a0b6ab42820da6eb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:23.891069 1259 raft_consensus.cc:482] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:23.891577 1259 raft_consensus.cc:504] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.893654 1259 leader_election.cc:288] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:23.894268 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88644048fa4a443fac09e951d78b3063" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:23.894522 32099 leader_election.cc:302] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.895367 1263 raft_consensus.cc:2726] T 8f35f6dc9a33438db135d9978caf3b14 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:30:23.895597 32439 leader_election.cc:341] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.896811 32099 leader_election.cc:302] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.897459 1263 raft_consensus.cc:2726] T b6f17af7f81741c68bafa73c2c3d91af P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:23.901008 32099 leader_election.cc:302] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.900290 1281 tablet.cc:1722] T d885baba6d6045e8b1ebcab6513719bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:23.901790 1263 raft_consensus.cc:2726] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:23.903290 32099 leader_election.cc:302] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.904237 1263 raft_consensus.cc:2726] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:23.913223 1263 raft_consensus.cc:482] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:23.913823 32691 tablet_bootstrap.cc:492] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:23.913616 1263 raft_consensus.cc:504] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.914255 32691 ts_tablet_manager.cc:1397] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.394s user 0.012s sys 0.008s
I20260809 14:30:23.919833 1263 leader_election.cc:288] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:23.921617 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:23.930200 32508 raft_consensus.cc:2379] T f74b4deeb32941aeaa3c4b813333c14e P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in this term.
I20260809 14:30:23.940377 32099 leader_election.cc:302] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.941370 1263 raft_consensus.cc:2726] T 67afcb47ffc947c5b1d98c96de333116 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:23.946189 32099 leader_election.cc:302] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.946998 1263 raft_consensus.cc:2726] T a051520649c84898ae634f58f3fb42b3 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:23.948127 32099 leader_election.cc:302] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.948879 1263 raft_consensus.cc:2726] T 5ff78e5cb7eb4796a8afa4847f0d2834 P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:30:23.950155 32099 leader_election.cc:341] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: Tablet not RUNNING: STOPPED
I20260809 14:30:23.950693 32099 leader_election.cc:302] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.952710 1239 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 852dbd0962ae4211923851c7a632768e)
I20260809 14:30:23.953154 1239 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:23.955492 1239 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:30:23.957082 819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 8 candidate_status { last_received { term: 7 index: 476 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:30:23.957710 819 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee93a411a1b04d38998018f8c6be59e0 in term 7.
I20260809 14:30:23.958927 31930 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0; no voters:
I20260809 14:30:23.959739 1239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 14:30:23.960143 1239 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 852dbd0962ae4211923851c7a632768e)
I20260809 14:30:23.960882 1239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 7 FOLLOWER]: Advancing to term 8
I20260809 14:30:23.970817 1239 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
W20260809 14:30:23.974140 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 455100us
I20260809 14:30:23.983620 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_83c723c78d1b466f96b992e07866b7e0 [id=345fabe93b8b4d038d0faf2de4dc8556]
I20260809 14:30:23.985901 1263 raft_consensus.cc:2726] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:23.988402 32099 leader_election.cc:302] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:23.989331 1263 raft_consensus.cc:2726] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 14:30:23.875490 1286 tablet.cc:1722] T d9f01d25eac34d4d998dc457be6a5d9b P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:23.881703 32333 raft_consensus.cc:3032] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Stepping down as leader of term 1
I20260809 14:30:23.993045 32333 raft_consensus.cc:729] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:23.993675 1020 log.cc:1196] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/94c4cbdda4ed48a8b34728d231497d02
I20260809 14:30:23.993830 32333 consensus_queue.cc:260] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:23.994819 32333 raft_consensus.cc:3037] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:23.995214 1020 ts_tablet_manager.cc:1939] T 94c4cbdda4ed48a8b34728d231497d02 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:23.884148 1261 raft_consensus.cc:2726] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:30:23.998086 32443 leader_election.cc:341] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.998764 32443 leader_election.cc:302] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:23.885850 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 67406.1us
I20260809 14:30:23.886023 32325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fa25dbd78f64e5db9e31d433e8f4736" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:24.002033 32260 leader_election.cc:341] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.887003 1285 tablet.cc:1722] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260809 14:30:24.009522 32324 raft_consensus.cc:2454] T 5ac7afd531a746cab7618ead98b4c4d9 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bed14aff53a41b2bbf5a13ce55dec5a in term 1.
I20260809 14:30:24.010300 1226 tablet.cc:1722] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:23.938366 1233 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47010 (request call id 267) took 5460 ms. Trace:
I20260809 14:30:24.011786 1233 rpcz_store.cc:276] 0809 14:30:18.478268 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.479545 (+ 1277us) service_pool.cc:227] Handling call
0809 14:30:23.938298 (+5458753us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:18.793370 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:18.793966 (+ 596us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:18.794030 (+ 64us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:18.794122 (+ 92us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:18.794272 (+ 150us) alter_schema_op.cc:126] START. Timestamp: 1786285818794140
0809 14:30:18.794319 (+ 47us) op_driver.cc:331] REPLICATION: starting
0809 14:30:18.795246 (+ 927us) log.cc:842] Serialized 173 byte log entry
0809 14:30:22.621558 (+3826312us) op_driver.cc:444] REPLICATION: finished
0809 14:30:23.092738 (+471180us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:23.094139 (+ 1401us) log.cc:842] Serialized 5 byte log entry
0809 14:30:23.867176 (+773037us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.937385 (+ 70209us) log.cc:842] Serialized 14 byte log entry
0809 14:30:23.938171 (+ 786us) alter_schema_op.cc:71] Released schema lock
0809 14:30:23.938197 (+ 26us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":470954,"mutex_wait_us":69355,"prepare.queue_time_us":312997,"prepare.run_cpu_time_us":27163,"prepare.run_wall_time_us":747249,"replication_time_us":3827115,"thread_start_us":1655689,"threads_started":4,"wal-append.queue_time_us":858598,"wal-append.run_cpu_time_us":2951,"wal-append.run_wall_time_us":1344324}]]}
I20260809 14:30:23.984444 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" candidate_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:24.014724 32331 raft_consensus.cc:2379] T f74b4deeb32941aeaa3c4b813333c14e P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1f94e0d792c46c9a0b6ab42820da6eb in current term 1: Already voted for candidate 3eb0bc287b12421d89abf97ca66e4d9b in this term.
I20260809 14:30:24.020092 32691 raft_consensus.cc:348] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.020962 32691 raft_consensus.cc:374] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:24.021279 32691 raft_consensus.cc:729] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:24.021946 32691 consensus_queue.cc:260] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:24.023164 32260 leader_election.cc:341] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:24.024020 1264 raft_consensus.cc:2726] T 3d81046d929c48a3a6946f20d8fd7bca P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:24.026908 1264 consensus_queue.cc:1035] T 67afcb47ffc947c5b1d98c96de333116 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260809 14:30:24.028903 32443 leader_election.cc:341] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:23.952417 32693 tablet_bootstrap.cc:654] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:24.037916 1020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 245) took 8051 ms. Trace:
W20260809 14:30:24.039216 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 50916.5us
W20260809 14:30:24.040879 32099 leader_election.cc:341] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Not found: Tablet not found: 2f171260cb764ac284aef85e4219767e
I20260809 14:30:24.041416 32099 leader_election.cc:302] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:30:24.041998 32099 raft_consensus.cc:2670] T 2f171260cb764ac284aef85e4219767e P e1f94e0d792c46c9a0b6ab42820da6eb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 14:30:24.084153 819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee93a411a1b04d38998018f8c6be59e0" candidate_term: 8 candidate_status { last_received { term: 7 index: 476 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3"
I20260809 14:30:24.084936 819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 14:30:23.986797 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_af577ceac58c4c31bfbaa3919493d8a2 [id=350bc1b7fbbe4d858f15b7a8003f069f]
I20260809 14:30:24.092197 819 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee93a411a1b04d38998018f8c6be59e0 in term 8.
I20260809 14:30:24.093490 31930 leader_election.cc:302] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d10512ac8b3406098fad1f7da012ab3, ee93a411a1b04d38998018f8c6be59e0; no voters:
I20260809 14:30:23.951803 32705 tablet_bootstrap.cc:654] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:24.103952 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:24.114441 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5121e9f5a85a45d285d78f41a442769a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:24.068025 32508 raft_consensus.cc:1226] T 43eeed4a20d6413f873585c95bb2858a P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Rejecting Update request from peer 3eb0bc287b12421d89abf97ca66e4d9b for earlier term 1. Current term is 2. Ops: [1.2-1.2]
I20260809 14:30:24.104640 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1af25739774df8a5536c75c2e1f416" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:24.117388 32443 leader_election.cc:302] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:24.118197 1264 raft_consensus.cc:2726] T 6fa25dbd78f64e5db9e31d433e8f4736 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:24.119369 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 90494.7us
I20260809 14:30:24.120445 32443 leader_election.cc:302] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:24.121444 1264 raft_consensus.cc:2781] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:24.121906 1264 raft_consensus.cc:686] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 LEADER]: Becoming Leader. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Running, Role: LEADER
I20260809 14:30:24.122676 1264 consensus_queue.cc:237] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:24.125468 31931 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet c423cc085ce9488697bb63159228541f (table default.Table_eaa055a1bfde40768487fe94df03d134 [id=4ff5e70bfad84d4b88d5966188c69279], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c423cc085ce9488697bb63159228541f consensus state has no leader
I20260809 14:30:24.128048 32508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5121e9f5a85a45d285d78f41a442769a" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:24.112821 1239 leader_election.cc:288] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 8 election: Requested vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169)
I20260809 14:30:24.134580 32165 raft_consensus.cc:1226] T 8b53f7d398d64701b4504db5c9c440b8 P e1f94e0d792c46c9a0b6ab42820da6eb [term 2 FOLLOWER]: Rejecting Update request from peer 3eb0bc287b12421d89abf97ca66e4d9b for earlier term 1. Current term is 2. Ops: []
I20260809 14:30:24.050134 32333 raft_consensus.cc:1261] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Refusing update from remote peer 6bed14aff53a41b2bbf5a13ce55dec5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 2. (term mismatch)
I20260809 14:30:24.051163 1233 tablet.cc:1722] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260809 14:30:24.136876 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 95965.8us
W20260809 14:30:24.072800 32398 tablet.cc:2369] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 14:30:24.139626 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_202a2a6d9c0b42c2940026fafa15fafe [id=38c58634369648909e70d249f01651a6]
I20260809 14:30:24.104652 32691 ts_tablet_manager.cc:1428] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.190s user 0.007s sys 0.000s
I20260809 14:30:24.137319 32333 pending_rounds.cc:76] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting all ops after (but not including) 1
I20260809 14:30:24.154191 32691 tablet_bootstrap.cc:492] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
W20260809 14:30:24.158555 31930 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 65050.6us
I20260809 14:30:24.129662 1239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 14:30:24.165690 32099 leader_election.cc:302] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
W20260809 14:30:24.167887 32099 raft_consensus.cc:2670] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 14:30:24.171603 31945 catalog_manager.cc:6530] Service unavailable: an error occurred while writing to the sys-catalog: leader is not yet ready
I20260809 14:30:24.174698 1251 tablet.cc:1722] T d885baba6d6045e8b1ebcab6513719bb P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:24.175225 31945 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49782 (request call id 438) took 2037 ms. Trace:
I20260809 14:30:24.175499 31945 rpcz_store.cc:276] 0809 14:30:22.137772 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.138636 (+ 864us) service_pool.cc:227] Handling call
0809 14:30:24.171288 (+2032652us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:24.171308 (+ 20us) write_op.cc:454] Released schema lock
0809 14:30:24.175136 (+ 3828us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:24.171058 (+ 0us) op_driver.cc:372] HandleFailure(Service unavailable: leader is not yet ready)
0809 14:30:24.171104 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:24.171122 (+ 18us) write_op.cc:454] Released schema lock
0809 14:30:24.171142 (+ 20us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{}]]}
F20260809 14:30:24.178124 32599 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Service unavailable: an error occurred while writing to the sys-catalog: leader is not yet ready
I20260809 14:30:24.181345 1298 tablet.cc:1722] T 43eeed4a20d6413f873585c95bb2858a P e1f94e0d792c46c9a0b6ab42820da6eb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:24.172525 1239 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 8 LEADER]: Becoming Leader. State: Replica: ee93a411a1b04d38998018f8c6be59e0, State: Running, Role: LEADER
*** Check failure stack trace: ***
*** Aborted at 1786285824 (unix time) try "date -d @1786285824" if you are using GNU date ***
W20260809 14:30:24.174486 31940 catalog_manager.cc:6530] Service unavailable: an error occurred while writing to the sys-catalog: leader is not yet ready
I20260809 14:30:24.182601 1239 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 476, Committed index: 476, Last appended: 7.476, Last appended by leader: 476, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:24.184898 31940 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:49782 (request call id 439) took 1452 ms. Trace:
I20260809 14:30:24.185393 31940 rpcz_store.cc:276] 0809 14:30:22.733026 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.733239 (+ 213us) service_pool.cc:227] Handling call
0809 14:30:24.174284 (+1441045us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:24.174305 (+ 21us) write_op.cc:454] Released schema lock
0809 14:30:24.184841 (+ 10536us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:24.174121 (+ 0us) op_driver.cc:372] HandleFailure(Service unavailable: leader is not yet ready)
0809 14:30:24.174152 (+ 31us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:24.174170 (+ 18us) write_op.cc:454] Released schema lock
0809 14:30:24.174186 (+ 16us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{}]]}
F20260809 14:30:24.187188 32595 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Service unavailable: an error occurred while writing to the sys-catalog: leader is not yet ready
*** Check failure stack trace: ***
I20260809 14:30:24.137104 1020 rpcz_store.cc:276] 0809 14:30:15.986970 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:15.988705 (+ 1735us) service_pool.cc:227] Handling call
0809 14:30:22.641288 (+6652583us) ts_tablet_manager.cc:1145] Deleting tablet 94c4cbdda4ed48a8b34728d231497d02
0809 14:30:22.641356 (+ 68us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:23.459566 (+818210us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:23.860504 (+400938us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.037856 (+177352us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":260437,"tablet-delete.queue_time_us":6647183}
I20260809 14:30:24.194186 32507 raft_consensus.cc:1226] T b6f17af7f81741c68bafa73c2c3d91af P 6bed14aff53a41b2bbf5a13ce55dec5a [term 2 FOLLOWER]: Rejecting Update request from peer 3eb0bc287b12421d89abf97ca66e4d9b for earlier term 1. Current term is 2. Ops: []
I20260809 14:30:24.196597 31943 catalog_manager.cc:5495] T 32cb7842617a4779bf110e4ea254e011 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:24.198101 31943 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 32cb7842617a4779bf110e4ea254e011 (table default.Table_423d2129e8fa4a59a8b4a7e853467ffa [id=2d493ef1e83d47da9117ecd5312dacdc]). Expected version 2 got 0
I20260809 14:30:24.198966 31943 catalog_manager.cc:5495] T 88644048fa4a443fac09e951d78b3063 P 3eb0bc287b12421d89abf97ca66e4d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66). New cstate: current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260809 14:30:24.200374 31943 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 88644048fa4a443fac09e951d78b3063 (table default.Table_569de184b14243abb58d1414941be5aa [id=67767d6db83f4ae6ab265b854d2ad695]). Expected version 2 got 0
I20260809 14:30:24.201264 31943 catalog_manager.cc:5587] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891) does not have the latest schema for tablet 43eeed4a20d6413f873585c95bb2858a (table default.Table_d6b92d6131e247fdbeda2d0138963386 [id=8b074e08bcaf4547af80d456e4c0abdf]). Expected version 1 got 0
I20260809 14:30:20.304086 820 rpcz_store.cc:276] 0809 14:30:13.936056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:13.936326 (+ 270us) service_pool.cc:227] Handling call
0809 14:30:14.833268 (+896942us) raft_consensus.cc:1457] Updating replica for 309 ops
0809 14:30:15.102568 (+269300us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 14:30:15.102614 (+ 46us) raft_consensus.cc:1525] Triggering prepare for 309 ops
0809 14:30:15.103590 (+ 976us) write_op.cc:270] Start()
0809 14:30:15.103865 (+ 275us) write_op.cc:276] Timestamp: P: 1786285793040272 usec, L: 0
0809 14:30:15.149733 (+ 45868us) write_op.cc:270] Start()
0809 14:30:15.149838 (+ 105us) write_op.cc:276] Timestamp: P: 1786285793532008 usec, L: 0
0809 14:30:15.150771 (+ 933us) write_op.cc:270] Start()
0809 14:30:15.150968 (+ 197us) write_op.cc:276] Timestamp: P: 1786285793664213 usec, L: 0
0809 14:30:15.151871 (+ 903us) write_op.cc:270] Start()
0809 14:30:15.152009 (+ 138us) write_op.cc:276] Timestamp: P: 1786285793733829 usec, L: 0
0809 14:30:15.152941 (+ 932us) write_op.cc:270] Start()
0809 14:30:15.153069 (+ 128us) write_op.cc:276] Timestamp: P: 1786285793759464 usec, L: 0
0809 14:30:15.153947 (+ 878us) write_op.cc:270] Start()
0809 14:30:15.154085 (+ 138us) write_op.cc:276] Timestamp: P: 1786285793768124 usec, L: 0
0809 14:30:15.155495 (+ 1410us) write_op.cc:270] Start()
0809 14:30:15.155697 (+ 202us) write_op.cc:276] Timestamp: P: 1786285793802415 usec, L: 0
0809 14:30:15.162522 (+ 6825us) write_op.cc:270] Start()
0809 14:30:15.162677 (+ 155us) write_op.cc:276] Timestamp: P: 1786285793878731 usec, L: 0
0809 14:30:15.165702 (+ 3025us) write_op.cc:270] Start()
0809 14:30:15.165831 (+ 129us) write_op.cc:276] Timestamp: P: 1786285793910295 usec, L: 0
0809 14:30:15.166721 (+ 890us) write_op.cc:270] Start()
0809 14:30:15.166830 (+ 109us) write_op.cc:276] Timestamp: P: 1786285793945119 usec, L: 0
0809 14:30:15.167462 (+ 632us) write_op.cc:270] Start()
0809 14:30:15.167577 (+ 115us) write_op.cc:276] Timestamp: P: 1786285793951999 usec, L: 0
0809 14:30:15.168220 (+ 643us) write_op.cc:270] Start()
0809 14:30:15.168352 (+ 132us) write_op.cc:276] Timestamp: P: 1786285794000521 usec, L: 0
0809 14:30:15.171596 (+ 3244us) write_op.cc:270] Start()
0809 14:30:15.171703 (+ 107us) write_op.cc:276] Timestamp: P: 1786285794114601 usec, L: 0
0809 14:30:15.172334 (+ 631us) write_op.cc:270] Start()
0809 14:30:15.172438 (+ 104us) write_op.cc:276] Timestamp: P: 1786285794127952 usec, L: 0
0809 14:30:15.198134 (+ 25696us) write_op.cc:270] Start()
0809 14:30:15.198283 (+ 149us) write_op.cc:276] Timestamp: P: 1786285794294368 usec, L: 0
0809 14:30:15.198997 (+ 714us) write_op.cc:270] Start()
0809 14:30:15.199105 (+ 108us) write_op.cc:276] Timestamp: P: 1786285794349711 usec, L: 0
0809 14:30:15.213510 (+ 14405us) write_op.cc:270] Start()
0809 14:30:15.215483 (+ 1973us) write_op.cc:276] Timestamp: P: 1786285794363942 usec, L: 0
0809 14:30:15.219110 (+ 3627us) write_op.cc:270] Start()
0809 14:30:15.220696 (+ 1586us) write_op.cc:276] Timestamp: P: 1786285794366047 usec, L: 0
0809 14:30:15.222530 (+ 1834us) write_op.cc:270] Start()
0809 14:30:15.222678 (+ 148us) write_op.cc:276] Timestamp: P: 1786285794368178 usec, L: 0
0809 14:30:15.223615 (+ 937us) write_op.cc:270] Start()
0809 14:30:15.223775 (+ 160us) write_op.cc:276] Timestamp: P: 1786285794395586 usec, L: 0
0809 14:30:15.228902 (+ 5127us) write_op.cc:270] Start()
0809 14:30:15.267488 (+ 38586us) write_op.cc:276] Timestamp: P: 1786285794453794 usec, L: 0
0809 14:30:15.268892 (+ 1404us) write_op.cc:270] Start()
0809 14:30:15.269134 (+ 242us) write_op.cc:276] Timestamp: P: 1786285794553957 usec, L: 0
0809 14:30:15.269708 (+ 574us) write_op.cc:270] Start()
0809 14:30:15.269809 (+ 101us) write_op.cc:276] Timestamp: P: 1786285794610968 usec, L: 0
0809 14:30:15.270279 (+ 470us) write_op.cc:270] Start()
0809 14:30:15.270441 (+ 162us) write_op.cc:276] Timestamp: P: 1786285794622760 usec, L: 0
0809 14:30:15.271056 (+ 615us) write_op.cc:270] Start()
0809 14:30:15.271137 (+ 81us) write_op.cc:276] Timestamp: P: 1786285794778293 usec, L: 0
0809 14:30:15.271582 (+ 445us) write_op.cc:270] Start()
0809 14:30:15.271676 (+ 94us) write_op.cc:276] Timestamp: P: 1786285794828603 usec, L: 0
0809 14:30:15.272108 (+ 432us) write_op.cc:270] Start()
0809 14:30:15.272179 (+ 71us) write_op.cc:276] Timestamp: P: 1786285794863388 usec, L: 0
0809 14:30:15.273658 (+ 1479us) write_op.cc:270] Start()
0809 14:30:15.273744 (+ 86us) write_op.cc:276] Timestamp: P: 1786285794903167 usec, L: 0
0809 14:30:15.279016 (+ 5272us) write_op.cc:270] Start()
0809 14:30:15.279131 (+ 115us) write_op.cc:276] Timestamp: P: 1786285794932809 usec, L: 0
0809 14:30:15.279706 (+ 575us) write_op.cc:270] Start()
0809 14:30:15.279829 (+ 123us) write_op.cc:276] Timestamp: P: 1786285794956978 usec, L: 0
0809 14:30:15.280353 (+ 524us) write_op.cc:270] Start()
0809 14:30:15.280473 (+ 120us) write_op.cc:276] Timestamp: P: 1786285795018582 usec, L: 0
0809 14:30:15.280984 (+ 511us) write_op.cc:270] Start()
0809 14:30:15.281077 (+ 93us) write_op.cc:276] Timestamp: P: 1786285795058577 usec, L: 0
0809 14:30:15.281688 (+ 611us) write_op.cc:270] Start()
0809 14:30:15.281784 (+ 96us) write_op.cc:276] Timestamp: P: 1786285795074558 usec, L: 0
0809 14:30:15.282292 (+ 508us) write_op.cc:270] Start()
0809 14:30:15.282386 (+ 94us) write_op.cc:276] Timestamp: P: 1786285795079493 usec, L: 0
0809 14:30:15.282905 (+ 519us) write_op.cc:270] Start()
0809 14:30:15.283000 (+ 95us) write_op.cc:276] Timestamp: P: 1786285795134320 usec, L: 0
0809 14:30:15.298506 (+ 15506us) write_op.cc:270] Start()
0809 14:30:15.298639 (+ 133us) write_op.cc:276] Timestamp: P: 1786285795164205 usec, L: 0
0809 14:30:15.299208 (+ 569us) write_op.cc:270] Start()
0809 14:30:15.299315 (+ 107us) write_op.cc:276] Timestamp: P: 1786285795187436 usec, L: 0
0809 14:30:15.301034 (+ 1719us) write_op.cc:270] Start()
0809 14:30:15.301138 (+ 104us) write_op.cc:276] Timestamp: P: 1786285795211271 usec, L: 0
0809 14:30:15.301694 (+ 556us) write_op.cc:270] Start()
0809 14:30:15.301789 (+ 95us) write_op.cc:276] Timestamp: P: 1786285795236029 usec, L: 0
0809 14:30:15.303781 (+ 1992us) write_op.cc:270] Start()
0809 14:30:15.303883 (+ 102us) write_op.cc:276] Timestamp: P: 1786285795260720 usec, L: 0
0809 14:30:15.309782 (+ 5899us) write_op.cc:270] Start()
0809 14:30:15.309901 (+ 119us) write_op.cc:276] Timestamp: P: 1786285795340053 usec, L: 0
0809 14:30:15.310990 (+ 1089us) write_op.cc:270] Start()
0809 14:30:15.311085 (+ 95us) write_op.cc:276] Timestamp: P: 1786285795387195 usec, L: 0
0809 14:30:15.315674 (+ 4589us) write_op.cc:270] Start()
0809 14:30:15.315796 (+ 122us) write_op.cc:276] Timestamp: P: 1786285795408282 usec, L: 0
0809 14:30:15.316365 (+ 569us) write_op.cc:270] Start()
0809 14:30:15.332769 (+ 16404us) write_op.cc:276] Timestamp: P: 1786285795444973 usec, L: 0
0809 14:30:15.333519 (+ 750us) write_op.cc:270] Start()
0809 14:30:15.333639 (+ 120us) write_op.cc:276] Timestamp: P: 1786285795489010 usec, L: 0
0809 14:30:15.349305 (+ 15666us) write_op.cc:270] Start()
0809 14:30:15.349405 (+ 100us) write_op.cc:276] Timestamp: P: 1786285795538566 usec, L: 0
0809 14:30:15.349984 (+ 579us) write_op.cc:270] Start()
0809 14:30:15.350076 (+ 92us) write_op.cc:276] Timestamp: P: 1786285795631455 usec, L: 0
0809 14:30:15.350553 (+ 477us) write_op.cc:270] Start()
0809 14:30:15.350702 (+ 149us) write_op.cc:276] Timestamp: P: 1786285795634476 usec, L: 0
0809 14:30:15.351223 (+ 521us) write_op.cc:270] Start()
0809 14:30:15.351300 (+ 77us) write_op.cc:276] Timestamp: P: 1786285795636374 usec, L: 0
0809 14:30:15.351756 (+ 456us) write_op.cc:270] Start()
0809 14:30:15.351829 (+ 73us) write_op.cc:276] Timestamp: P: 1786285795709926 usec, L: 0
0809 14:30:15.352333 (+ 504us) write_op.cc:270] Start()
0809 14:30:15.378523 (+ 26190us) write_op.cc:276] Timestamp: P: 1786285795718572 usec, L: 0
0809 14:30:15.389550 (+ 11027us) write_op.cc:270] Start()
0809 14:30:15.389774 (+ 224us) write_op.cc:276] Timestamp: P: 1786285795753434 usec, L: 0
0809 14:30:15.425222 (+ 35448us) write_op.cc:270] Start()
0809 14:30:15.440880 (+ 15658us) write_op.cc:276] Timestamp: P: 1786285795793293 usec, L: 0
0809 14:30:15.442019 (+ 1139us) write_op.cc:270] Start()
0809 14:30:15.442207 (+ 188us) write_op.cc:276] Timestamp: P: 1786285795802237 usec, L: 0
0809 14:30:15.443221 (+ 1014us) write_op.cc:270] Start()
0809 14:30:15.473613 (+ 30392us) write_op.cc:276] Timestamp: P: 1786285795838534 usec, L: 0
0809 14:30:15.480676 (+ 7063us) write_op.cc:270] Start()
0809 14:30:15.480825 (+ 149us) write_op.cc:276] Timestamp: P: 1786285795909752 usec, L: 0
0809 14:30:15.500558 (+ 19733us) write_op.cc:270] Start()
0809 14:30:15.500757 (+ 199us) write_op.cc:276] Timestamp: P: 1786285795930337 usec, L: 0
0809 14:30:15.508442 (+ 7685us) write_op.cc:270] Start()
0809 14:30:15.510707 (+ 2265us) write_op.cc:276] Timestamp: P: 1786285795969974 usec, L: 0
0809 14:30:15.534739 (+ 24032us) write_op.cc:270] Start()
0809 14:30:15.534862 (+ 123us) write_op.cc:276] Timestamp: P: 1786285795991416 usec, L: 0
0809 14:30:15.535536 (+ 674us) write_op.cc:270] Start()
0809 14:30:15.535811 (+ 275us) write_op.cc:276] Timestamp: P: 1786285796102910 usec, L: 0
0809 14:30:15.580973 (+ 45162us) write_op.cc:270] Start()
0809 14:30:15.581206 (+ 233us) write_op.cc:276] Timestamp: P: 1786285796111948 usec, L: 0
0809 14:30:15.582319 (+ 1113us) write_op.cc:270] Start()
0809 14:30:15.582451 (+ 132us) write_op.cc:276] Timestamp: P: 1786285796114016 usec, L: 0
0809 14:30:15.583558 (+ 1107us) write_op.cc:270] Start()
0809 14:30:15.583791 (+ 233us) write_op.cc:276] Timestamp: P: 1786285796115972 usec, L: 0
0809 14:30:15.612216 (+ 28425us) write_op.cc:270] Start()
0809 14:30:15.612333 (+ 117us) write_op.cc:276] Timestamp: P: 1786285796144723 usec, L: 0
0809 14:30:15.613170 (+ 837us) write_op.cc:270] Start()
0809 14:30:15.613283 (+ 113us) write_op.cc:276] Timestamp: P: 1786285796213755 usec, L: 0
0809 14:30:15.613959 (+ 676us) write_op.cc:270] Start()
0809 14:30:15.614069 (+ 110us) write_op.cc:276] Timestamp: P: 1786285796217452 usec, L: 0
0809 14:30:15.627714 (+ 13645us) write_op.cc:270] Start()
0809 14:30:15.627816 (+ 102us) write_op.cc:276] Timestamp: P: 1786285796354342 usec, L: 0
0809 14:30:15.628355 (+ 539us) write_op.cc:270] Start()
0809 14:30:15.628435 (+ 80us) write_op.cc:276] Timestamp: P: 1786285796356385 usec, L: 0
0809 14:30:15.628919 (+ 484us) write_op.cc:270] Start()
0809 14:30:15.628993 (+ 74us) write_op.cc:276] Timestamp: P: 1786285796375266 usec, L: 0
0809 14:30:15.629443 (+ 450us) write_op.cc:270] Start()
0809 14:30:15.629542 (+ 99us) write_op.cc:276] Timestamp: P: 1786285796406300 usec, L: 0
0809 14:30:15.630021 (+ 479us) write_op.cc:270] Start()
0809 14:30:15.630103 (+ 82us) write_op.cc:276] Timestamp: P: 1786285796420241 usec, L: 0
0809 14:30:15.630568 (+ 465us) write_op.cc:270] Start()
0809 14:30:15.630646 (+ 78us) write_op.cc:276] Timestamp: P: 1786285796635829 usec, L: 0
0809 14:30:15.645787 (+ 15141us) write_op.cc:270] Start()
0809 14:30:15.645908 (+ 121us) write_op.cc:276] Timestamp: P: 1786285796639315 usec, L: 0
0809 14:30:15.646478 (+ 570us) write_op.cc:270] Start()
0809 14:30:15.646585 (+ 107us) write_op.cc:276] Timestamp: P: 1786285796646232 usec, L: 0
0809 14:30:15.647046 (+ 461us) write_op.cc:270] Start()
0809 14:30:15.647152 (+ 106us) write_op.cc:276] Timestamp: P: 1786285796649842 usec, L: 0
0809 14:30:15.647707 (+ 555us) write_op.cc:270] Start()
0809 14:30:15.647796 (+ 89us) write_op.cc:276] Timestamp: P: 1786285796729352 usec, L: 0
0809 14:30:15.648283 (+ 487us) write_op.cc:270] Start()
0809 14:30:15.648374 (+ 91us) write_op.cc:276] Timestamp: P: 1786285796787287 usec, L: 0
0809 14:30:15.658620 (+ 10246us) write_op.cc:270] Start()
0809 14:30:15.658739 (+ 119us) write_op.cc:276] Timestamp: P: 1786285796931135 usec, L: 0
0809 14:30:15.659285 (+ 546us) write_op.cc:270] Start()
0809 14:30:15.659385 (+ 100us) write_op.cc:276] Timestamp: P: 1786285796956081 usec, L: 0
0809 14:30:15.659910 (+ 525us) write_op.cc:270] Start()
0809 14:30:15.660001 (+ 91us) write_op.cc:276] Timestamp: P: 1786285797147168 usec, L: 0
0809 14:30:15.660505 (+ 504us) write_op.cc:270] Start()
0809 14:30:15.832848 (+172343us) write_op.cc:276] Timestamp: P: 1786285797162107 usec, L: 0
0809 14:30:15.833570 (+ 722us) write_op.cc:270] Start()
0809 14:30:15.833678 (+ 108us) write_op.cc:276] Timestamp: P: 1786285797189513 usec, L: 0
0809 14:30:15.834238 (+ 560us) write_op.cc:270] Start()
0809 14:30:15.834324 (+ 86us) write_op.cc:276] Timestamp: P: 1786285797195957 usec, L: 0
0809 14:30:15.834821 (+ 497us) write_op.cc:270] Start()
0809 14:30:15.834933 (+ 112us) write_op.cc:276] Timestamp: P: 1786285797273597 usec, L: 0
0809 14:30:15.835468 (+ 535us) write_op.cc:270] Start()
0809 14:30:15.835549 (+ 81us) write_op.cc:276] Timestamp: P: 1786285797276596 usec, L: 0
0809 14:30:15.988089 (+152540us) write_op.cc:270] Start()
0809 14:30:15.988228 (+ 139us) write_op.cc:276] Timestamp: P: 1786285797395038 usec, L: 0
0809 14:30:15.989229 (+ 1001us) write_op.cc:270] Start()
0809 14:30:15.989465 (+ 236us) write_op.cc:276] Timestamp: P: 1786285797479495 usec, L: 0
0809 14:30:15.990635 (+ 1170us) write_op.cc:270] Start()
0809 14:30:16.002080 (+ 11445us) write_op.cc:276] Timestamp: P: 1786285797570257 usec, L: 0
0809 14:30:16.002667 (+ 587us) write_op.cc:270] Start()
0809 14:30:16.002755 (+ 88us) write_op.cc:276] Timestamp: P: 1786285797604211 usec, L: 0
0809 14:30:16.030483 (+ 27728us) write_op.cc:270] Start()
0809 14:30:16.030589 (+ 106us) write_op.cc:276] Timestamp: P: 1786285797636202 usec, L: 0
0809 14:30:16.045570 (+ 14981us) write_op.cc:270] Start()
0809 14:30:16.045679 (+ 109us) write_op.cc:276] Timestamp: P: 1786285797649716 usec, L: 0
0809 14:30:16.046249 (+ 570us) write_op.cc:270] Start()
0809 14:30:16.046338 (+ 89us) write_op.cc:276] Timestamp: P: 1786285797653726 usec, L: 0
0809 14:30:16.046810 (+ 472us) write_op.cc:270] Start()
0809 14:30:16.046904 (+ 94us) write_op.cc:276] Timestamp: P: 1786285797690444 usec, L: 0
0809 14:30:16.047476 (+ 572us) write_op.cc:270] Start()
0809 14:30:16.047565 (+ 89us) write_op.cc:276] Timestamp: P: 1786285797709646 usec, L: 0
0809 14:30:16.048095 (+ 530us) write_op.cc:270] Start()
0809 14:30:16.048186 (+ 91us) write_op.cc:276] Timestamp: P: 1786285797752532 usec, L: 0
0809 14:30:16.103925 (+ 55739us) write_op.cc:270] Start()
0809 14:30:16.104036 (+ 111us) write_op.cc:276] Timestamp: P: 1786285797765511 usec, L: 0
0809 14:30:16.104686 (+ 650us) write_op.cc:270] Start()
0809 14:30:16.104772 (+ 86us) write_op.cc:276] Timestamp: P: 1786285797801053 usec, L: 0
0809 14:30:16.105247 (+ 475us) write_op.cc:270] Start()
0809 14:30:16.105453 (+ 206us) write_op.cc:276] Timestamp: P: 1786285797887715 usec, L: 0
0809 14:30:16.106023 (+ 570us) write_op.cc:270] Start()
0809 14:30:16.106165 (+ 142us) write_op.cc:276] Timestamp: P: 1786285798141089 usec, L: 0
0809 14:30:16.106611 (+ 446us) write_op.cc:270] Start()
0809 14:30:16.106688 (+ 77us) write_op.cc:276] Timestamp: P: 1786285798294968 usec, L: 0
0809 14:30:16.107155 (+ 467us) write_op.cc:270] Start()
0809 14:30:16.171060 (+ 63905us) write_op.cc:276] Timestamp: P: 1786285798535670 usec, L: 0
0809 14:30:16.171683 (+ 623us) write_op.cc:270] Start()
0809 14:30:16.171780 (+ 97us) write_op.cc:276] Timestamp: P: 1786285798742031 usec, L: 0
0809 14:30:16.172234 (+ 454us) write_op.cc:270] Start()
0809 14:30:16.172310 (+ 76us) write_op.cc:276] Timestamp: P: 1786285798781706 usec, L: 0
0809 14:30:16.172824 (+ 514us) write_op.cc:270] Start()
0809 14:30:16.172904 (+ 80us) write_op.cc:276] Timestamp: P: 1786285798894140 usec, L: 0
0809 14:30:16.173383 (+ 479us) write_op.cc:270] Start()
0809 14:30:16.173461 (+ 78us) write_op.cc:276] Timestamp: P: 1786285798919793 usec, L: 0
0809 14:30:16.174149 (+ 688us) write_op.cc:270] Start()
0809 14:30:16.174222 (+ 73us) write_op.cc:276] Timestamp: P: 1786285798933789 usec, L: 0
0809 14:30:16.176774 (+ 2552us) write_op.cc:270] Start()
0809 14:30:16.176861 (+ 87us) write_op.cc:276] Timestamp: P: 1786285799084583 usec, L: 0
0809 14:30:16.278829 (+101968us) write_op.cc:270] Start()
0809 14:30:16.278969 (+ 140us) write_op.cc:276] Timestamp: P: 1786285799166286 usec, L: 0
0809 14:30:16.280748 (+ 1779us) write_op.cc:270] Start()
0809 14:30:16.280861 (+ 113us) write_op.cc:276] Timestamp: P: 1786285799182370 usec, L: 0
0809 14:30:16.281472 (+ 611us) write_op.cc:270] Start()
0809 14:30:16.281566 (+ 94us) write_op.cc:276] Timestamp: P: 1786285799204824 usec, L: 0
0809 14:30:16.282214 (+ 648us) write_op.cc:270] Start()
0809 14:30:16.282307 (+ 93us) write_op.cc:276] Timestamp: P: 1786285799298350 usec, L: 0
0809 14:30:16.282736 (+ 429us) write_op.cc:270] Start()
0809 14:30:16.282814 (+ 78us) write_op.cc:276] Timestamp: P: 1786285799395502 usec, L: 0
0809 14:30:16.283367 (+ 553us) write_op.cc:270] Start()
0809 14:30:16.283464 (+ 97us) write_op.cc:276] Timestamp: P: 1786285799399064 usec, L: 0
0809 14:30:16.354451 (+ 70987us) write_op.cc:270] Start()
0809 14:30:16.354571 (+ 120us) write_op.cc:276] Timestamp: P: 1786285799401770 usec, L: 0
0809 14:30:16.355232 (+ 661us) write_op.cc:270] Start()
0809 14:30:16.355343 (+ 111us) write_op.cc:276] Timestamp: P: 1786285799410173 usec, L: 0
0809 14:30:16.355887 (+ 544us) write_op.cc:270] Start()
0809 14:30:16.355989 (+ 102us) write_op.cc:276] Timestamp: P: 1786285799488330 usec, L: 0
0809 14:30:16.465412 (+109423us) write_op.cc:270] Start()
0809 14:30:16.465522 (+ 110us) write_op.cc:276] Timestamp: P: 1786285799524260 usec, L: 0
0809 14:30:16.467006 (+ 1484us) write_op.cc:270] Start()
0809 14:30:16.467387 (+ 381us) write_op.cc:276] Timestamp: P: 1786285799527197 usec, L: 0
0809 14:30:16.467921 (+ 534us) write_op.cc:270] Start()
0809 14:30:16.468020 (+ 99us) write_op.cc:276] Timestamp: P: 1786285799554382 usec, L: 0
0809 14:30:16.533150 (+ 65130us) write_op.cc:270] Start()
0809 14:30:16.533261 (+ 111us) write_op.cc:276] Timestamp: P: 1786285799598026 usec, L: 0
0809 14:30:16.533862 (+ 601us) write_op.cc:270] Start()
0809 14:30:16.533969 (+ 107us) write_op.cc:276] Timestamp: P: 1786285799614921 usec, L: 0
0809 14:30:16.534423 (+ 454us) write_op.cc:270] Start()
0809 14:30:16.534499 (+ 76us) write_op.cc:276] Timestamp: P: 1786285799646792 usec, L: 0
0809 14:30:16.534963 (+ 464us) write_op.cc:270] Start()
0809 14:30:16.535045 (+ 82us) write_op.cc:276] Timestamp: P: 1786285799651868 usec, L: 0
0809 14:30:16.535468 (+ 423us) write_op.cc:270] Start()
0809 14:30:16.535541 (+ 73us) write_op.cc:276] Timestamp: P: 1786285799680931 usec, L: 0
0809 14:30:16.535975 (+ 434us) write_op.cc:270] Start()
0809 14:30:16.536050 (+ 75us) write_op.cc:276] Timestamp: P: 1786285799693370 usec, L: 0
0809 14:30:16.598247 (+ 62197us) write_op.cc:270] Start()
0809 14:30:16.598365 (+ 118us) write_op.cc:276] Timestamp: P: 1786285799698452 usec, L: 0
0809 14:30:16.599047 (+ 682us) write_op.cc:270] Start()
0809 14:30:16.599148 (+ 101us) write_op.cc:276] Timestamp: P: 1786285799796172 usec, L: 0
0809 14:30:16.599793 (+ 645us) write_op.cc:270] Start()
0809 14:30:16.599875 (+ 82us) write_op.cc:276] Timestamp: P: 1786285800043516 usec, L: 0
0809 14:30:16.601009 (+ 1134us) write_op.cc:270] Start()
0809 14:30:16.601107 (+ 98us) write_op.cc:276] Timestamp: P: 1786285800113710 usec, L: 0
0809 14:30:16.601678 (+ 571us) write_op.cc:270] Start()
0809 14:30:16.601761 (+ 83us) write_op.cc:276] Timestamp: P: 1786285800189704 usec, L: 0
0809 14:30:16.602243 (+ 482us) write_op.cc:270] Start()
0809 14:30:16.717752 (+115509us) write_op.cc:276] Timestamp: P: 1786285800261141 usec, L: 0
0809 14:30:16.741590 (+ 23838us) write_op.cc:270] Start()
0809 14:30:16.741686 (+ 96us) write_op.cc:276] Timestamp: P: 1786285800394549 usec, L: 0
0809 14:30:16.742398 (+ 712us) write_op.cc:270] Start()
0809 14:30:16.742478 (+ 80us) write_op.cc:276] Timestamp: P: 1786285800710171 usec, L: 0
0809 14:30:16.743023 (+ 545us) write_op.cc:270] Start()
0809 14:30:16.743091 (+ 68us) write_op.cc:276] Timestamp: P: 1786285800713753 usec, L: 0
0809 14:30:16.743632 (+ 541us) write_op.cc:270] Start()
0809 14:30:16.743704 (+ 72us) write_op.cc:276] Timestamp: P: 1786285800716211 usec, L: 0
0809 14:30:16.744417 (+ 713us) write_op.cc:270] Start()
0809 14:30:16.821144 (+ 76727us) write_op.cc:276] Timestamp: P: 1786285800884734 usec, L: 0
0809 14:30:16.821687 (+ 543us) write_op.cc:270] Start()
0809 14:30:16.821765 (+ 78us) write_op.cc:276] Timestamp: P: 1786285800898190 usec, L: 0
0809 14:30:16.822176 (+ 411us) write_op.cc:270] Start()
0809 14:30:16.822239 (+ 63us) write_op.cc:276] Timestamp: P: 1786285800960398 usec, L: 0
0809 14:30:16.822626 (+ 387us) write_op.cc:270] Start()
0809 14:30:16.822695 (+ 69us) write_op.cc:276] Timestamp: P: 1786285801021225 usec, L: 0
0809 14:30:16.823078 (+ 383us) write_op.cc:270] Start()
0809 14:30:16.823145 (+ 67us) write_op.cc:276] Timestamp: P: 1786285801105538 usec, L: 0
0809 14:30:16.823498 (+ 353us) write_op.cc:270] Start()
0809 14:30:16.823556 (+ 58us) write_op.cc:276] Timestamp: P: 1786285801140736 usec, L: 0
0809 14:30:16.823918 (+ 362us) write_op.cc:270] Start()
0809 14:30:16.823975 (+ 57us) write_op.cc:276] Timestamp: P: 1786285801232015 usec, L: 0
0809 14:30:16.824314 (+ 339us) write_op.cc:270] Start()
0809 14:30:16.824370 (+ 56us) write_op.cc:276] Timestamp: P: 1786285801258466 usec, L: 0
0809 14:30:16.893386 (+ 69016us) write_op.cc:270] Start()
0809 14:30:16.893469 (+ 83us) write_op.cc:276] Timestamp: P: 1786285801298760 usec, L: 0
0809 14:30:16.936408 (+ 42939us) write_op.cc:270] Start()
0809 14:30:16.936618 (+ 210us) write_op.cc:276] Timestamp: P: 1786285801426025 usec, L: 0
0809 14:30:16.949187 (+ 12569us) write_op.cc:270] Start()
0809 14:30:16.949279 (+ 92us) write_op.cc:276] Timestamp: P: 1786285801456902 usec, L: 0
0809 14:30:16.949820 (+ 541us) write_op.cc:270] Start()
0809 14:30:16.949914 (+ 94us) write_op.cc:276] Timestamp: P: 1786285801459646 usec, L: 0
0809 14:30:16.950442 (+ 528us) write_op.cc:270] Start()
0809 14:30:16.950526 (+ 84us) write_op.cc:276] Timestamp: P: 1786285801607716 usec, L: 0
0809 14:30:16.951132 (+ 606us) write_op.cc:270] Start()
0809 14:30:16.951210 (+ 78us) write_op.cc:276] Timestamp: P: 1786285801612204 usec, L: 0
0809 14:30:16.951731 (+ 521us) write_op.cc:270] Start()
0809 14:30:16.951827 (+ 96us) write_op.cc:276] Timestamp: P: 1786285801818361 usec, L: 0
0809 14:30:16.952224 (+ 397us) write_op.cc:270] Start()
0809 14:30:16.952296 (+ 72us) write_op.cc:276] Timestamp: P: 1786285802069576 usec, L: 0
0809 14:30:16.952839 (+ 543us) write_op.cc:270] Start()
0809 14:30:16.952907 (+ 68us) write_op.cc:276] Timestamp: P: 1786285802081365 usec, L: 0
0809 14:30:16.953368 (+ 461us) write_op.cc:270] Start()
0809 14:30:16.953436 (+ 68us) write_op.cc:276] Timestamp: P: 1786285802085993 usec, L: 0
0809 14:30:17.044952 (+ 91516us) write_op.cc:270] Start()
0809 14:30:17.045061 (+ 109us) write_op.cc:276] Timestamp: P: 1786285802088233 usec, L: 0
0809 14:30:17.045680 (+ 619us) write_op.cc:270] Start()
0809 14:30:17.045766 (+ 86us) write_op.cc:276] Timestamp: P: 1786285802090532 usec, L: 0
0809 14:30:17.046409 (+ 643us) write_op.cc:270] Start()
0809 14:30:17.046491 (+ 82us) write_op.cc:276] Timestamp: P: 1786285802311323 usec, L: 0
0809 14:30:17.046954 (+ 463us) write_op.cc:270] Start()
0809 14:30:17.047054 (+ 100us) write_op.cc:276] Timestamp: P: 1786285802323549 usec, L: 0
0809 14:30:17.048407 (+ 1353us) write_op.cc:270] Start()
0809 14:30:17.064169 (+ 15762us) write_op.cc:276] Timestamp: P: 1786285802344522 usec, L: 0
0809 14:30:17.064756 (+ 587us) write_op.cc:270] Start()
0809 14:30:17.064834 (+ 78us) write_op.cc:276] Timestamp: P: 1786285802349744 usec, L: 0
0809 14:30:17.065363 (+ 529us) write_op.cc:270] Start()
0809 14:30:17.065438 (+ 75us) write_op.cc:276] Timestamp: P: 1786285802351739 usec, L: 0
0809 14:30:17.066004 (+ 566us) write_op.cc:270] Start()
0809 14:30:17.066073 (+ 69us) write_op.cc:276] Timestamp: P: 1786285802393708 usec, L: 0
0809 14:30:17.066636 (+ 563us) write_op.cc:270] Start()
0809 14:30:17.066705 (+ 69us) write_op.cc:276] Timestamp: P: 1786285802421005 usec, L: 0
0809 14:30:17.067236 (+ 531us) write_op.cc:270] Start()
0809 14:30:17.067301 (+ 65us) write_op.cc:276] Timestamp: P: 1786285802422939 usec, L: 0
0809 14:30:17.067917 (+ 616us) write_op.cc:270] Start()
0809 14:30:17.067997 (+ 80us) write_op.cc:276] Timestamp: P: 1786285802424817 usec, L: 0
0809 14:30:17.103411 (+ 35414us) write_op.cc:270] Start()
0809 14:30:17.103531 (+ 120us) write_op.cc:276] Timestamp: P: 1786285802593157 usec, L: 0
0809 14:30:17.104148 (+ 617us) write_op.cc:270] Start()
0809 14:30:17.104222 (+ 74us) write_op.cc:276] Timestamp: P: 1786285802673114 usec, L: 0
0809 14:30:17.104938 (+ 716us) write_op.cc:270] Start()
0809 14:30:17.105015 (+ 77us) write_op.cc:276] Timestamp: P: 1786285802715639 usec, L: 0
0809 14:30:17.224636 (+119621us) write_op.cc:270] Start()
0809 14:30:17.224768 (+ 132us) write_op.cc:276] Timestamp: P: 1786285802817104 usec, L: 0
0809 14:30:17.225413 (+ 645us) write_op.cc:270] Start()
0809 14:30:17.225500 (+ 87us) write_op.cc:276] Timestamp: P: 1786285802823805 usec, L: 0
0809 14:30:17.225965 (+ 465us) write_op.cc:270] Start()
0809 14:30:17.226032 (+ 67us) write_op.cc:276] Timestamp: P: 1786285802886552 usec, L: 0
0809 14:30:17.226408 (+ 376us) write_op.cc:270] Start()
0809 14:30:17.226472 (+ 64us) write_op.cc:276] Timestamp: P: 1786285803256661 usec, L: 0
0809 14:30:17.226910 (+ 438us) write_op.cc:270] Start()
0809 14:30:17.226990 (+ 80us) write_op.cc:276] Timestamp: P: 1786285803327077 usec, L: 0
0809 14:30:17.227383 (+ 393us) write_op.cc:270] Start()
0809 14:30:17.227448 (+ 65us) write_op.cc:276] Timestamp: P: 1786285803329524 usec, L: 0
0809 14:30:17.227827 (+ 379us) write_op.cc:270] Start()
0809 14:30:17.227891 (+ 64us) write_op.cc:276] Timestamp: P: 1786285803331730 usec, L: 0
0809 14:30:17.228279 (+ 388us) write_op.cc:270] Start()
0809 14:30:17.228344 (+ 65us) write_op.cc:276] Timestamp: P: 1786285803444693 usec, L: 0
0809 14:30:17.256924 (+ 28580us) write_op.cc:270] Start()
0809 14:30:17.257016 (+ 92us) write_op.cc:276] Timestamp: P: 1786285803496726 usec, L: 0
0809 14:30:17.257496 (+ 480us) write_op.cc:270] Start()
0809 14:30:17.257567 (+ 71us) write_op.cc:276] Timestamp: P: 1786285803501527 usec, L: 0
0809 14:30:17.257968 (+ 401us) write_op.cc:270] Start()
0809 14:30:17.258042 (+ 74us) write_op.cc:276] Timestamp: P: 1786285803570379 usec, L: 0
0809 14:30:17.258443 (+ 401us) write_op.cc:270] Start()
0809 14:30:17.258515 (+ 72us) write_op.cc:276] Timestamp: P: 1786285803648005 usec, L: 0
0809 14:30:17.258907 (+ 392us) write_op.cc:270] Start()
0809 14:30:17.258971 (+ 64us) write_op.cc:276] Timestamp: P: 1786285803654206 usec, L: 0
0809 14:30:17.259364 (+ 393us) write_op.cc:270] Start()
0809 14:30:17.259428 (+ 64us) write_op.cc:276] Timestamp: P: 1786285803746248 usec, L: 0
0809 14:30:17.259820 (+ 392us) write_op.cc:270] Start()
0809 14:30:17.259889 (+ 69us) write_op.cc:276] Timestamp: P: 1786285803750303 usec, L: 0
0809 14:30:17.260307 (+ 418us) write_op.cc:270] Start()
0809 14:30:17.260386 (+ 79us) write_op.cc:276] Timestamp: P: 1786285803777690 usec, L: 0
0809 14:30:17.276967 (+ 16581us) write_op.cc:270] Start()
0809 14:30:17.277050 (+ 83us) write_op.cc:276] Timestamp: P: 1786285803789798 usec, L: 0
0809 14:30:17.277570 (+ 520us) write_op.cc:270] Start()
0809 14:30:17.277643 (+ 73us) write_op.cc:276] Timestamp: P: 1786285803989796 usec, L: 0
0809 14:30:17.278109 (+ 466us) write_op.cc:270] Start()
0809 14:30:17.278174 (+ 65us) write_op.cc:276] Timestamp: P: 1786285803995351 usec, L: 0
0809 14:30:17.278583 (+ 409us) write_op.cc:270] Start()
0809 14:30:17.278654 (+ 71us) write_op.cc:276] Timestamp: P: 1786285804061572 usec, L: 0
0809 14:30:17.279035 (+ 381us) write_op.cc:270] Start()
0809 14:30:17.279118 (+ 83us) write_op.cc:276] Timestamp: P: 1786285804147467 usec, L: 0
0809 14:30:17.279510 (+ 392us) write_op.cc:270] Start()
0809 14:30:17.279579 (+ 69us) write_op.cc:276] Timestamp: P: 1786285804227927 usec, L: 0
0809 14:30:17.279985 (+ 406us) write_op.cc:270] Start()
0809 14:30:17.301058 (+ 21073us) write_op.cc:276] Timestamp: P: 1786285804265294 usec, L: 0
0809 14:30:17.301565 (+ 507us) write_op.cc:270] Start()
0809 14:30:17.301635 (+ 70us) write_op.cc:276] Timestamp: P: 1786285804365825 usec, L: 0
0809 14:30:17.302029 (+ 394us) write_op.cc:270] Start()
0809 14:30:17.302107 (+ 78us) write_op.cc:276] Timestamp: P: 1786285804446952 usec, L: 0
0809 14:30:17.302583 (+ 476us) write_op.cc:270] Start()
0809 14:30:17.302650 (+ 67us) write_op.cc:276] Timestamp: P: 1786285804538730 usec, L: 0
0809 14:30:17.303190 (+ 540us) write_op.cc:270] Start()
0809 14:30:17.303258 (+ 68us) write_op.cc:276] Timestamp: P: 1786285804569790 usec, L: 0
0809 14:30:17.303810 (+ 552us) write_op.cc:270] Start()
0809 14:30:17.303882 (+ 72us) write_op.cc:276] Timestamp: P: 1786285804657720 usec, L: 0
0809 14:30:17.304379 (+ 497us) write_op.cc:270] Start()
0809 14:30:17.304506 (+ 127us) write_op.cc:276] Timestamp: P: 1786285804861656 usec, L: 0
0809 14:30:17.305020 (+ 514us) write_op.cc:270] Start()
0809 14:30:17.305092 (+ 72us) write_op.cc:276] Timestamp: P: 17862858
PC: @ 0x0 (unknown)
I20260809 14:30:24.172302 32099 leader_election.cc:302] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1f94e0d792c46c9a0b6ab42820da6eb; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a
I20260809 14:30:24.213654 1263 raft_consensus.cc:3037] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
W20260809 14:30:24.220280 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 99983.6us
I20260809 14:30:24.228806 1263 raft_consensus.cc:2726] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 14:30:24.233228 32099 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 61157.2us
I20260809 14:30:24.247224 31959 raft_consensus.cc:1226] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [term 8 LEADER]: Rejecting Update request from peer 852dbd0962ae4211923851c7a632768e for earlier term 7. Current term is 8. Ops: []
I20260809 14:30:24.250411 1294 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee93a411a1b04d38998018f8c6be59e0. Latest consensus state: current_term: 8 leader_uuid: "ee93a411a1b04d38998018f8c6be59e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } } }
I20260809 14:30:24.251153 1294 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [sys.catalog]: This master's current role is: LEADER
I20260809 14:30:24.252897 1227 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 }, Status: INVALID_TERM, Last received: 7.476, Next index: 477, Last known committed idx: 476, Time since last communication: 0.000s
I20260809 14:30:24.253834 1227 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [LEADER]: Peer ee93a411a1b04d38998018f8c6be59e0 log is divergent from this leader: its last log entry 8.477 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 476
I20260809 14:30:24.254300 1292 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 7 LEADER]: Stepping down as leader of term 7
I20260809 14:30:24.254591 1292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 852dbd0962ae4211923851c7a632768e, State: Running, Role: LEADER
I20260809 14:30:24.255323 1292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 476, Committed index: 476, Last appended: 7.476, Last appended by leader: 476, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
I20260809 14:30:24.256147 1292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 7 FOLLOWER]: Advancing to term 8
I20260809 14:30:24.266942 1168 raft_consensus.cc:482] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:24.269083 1168 raft_consensus.cc:504] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.270908 1168 leader_election.cc:288] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:24.271167 32331 raft_consensus.cc:3032] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Stepping down as leader of term 1
I20260809 14:30:24.271487 32331 raft_consensus.cc:729] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:24.272210 32331 consensus_queue.cc:260] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.277261 32331 raft_consensus.cc:3037] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Advancing to term 2
E20260809 14:30:24.298537 32031 catalog_manager.cc:816] Error processing pending assignments: Illegal state: error persisting updated tablet metadata: replica ee93a411a1b04d38998018f8c6be59e0 is not leader of this config: current role FOLLOWER
I20260809 14:30:24.156831 32333 pending_rounds.cc:97] T a051520649c84898ae634f58f3fb42b3 P 3eb0bc287b12421d89abf97ca66e4d9b: Aborting uncommitted ALTER_SCHEMA_OP operation due to leader change: 1.2
W20260809 14:30:24.305575 1291 log.cc:924] Time spent T 3d592b1c4520467791ffd9d36e0d97fd P 3eb0bc287b12421d89abf97ca66e4d9b: Append to log took a long time: real 0.167s user 0.001s sys 0.000s
I20260809 14:30:24.320186 32324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7856076c1332479a894d2d69d16b19f5" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:24.334692 1226 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.323s user 0.004s sys 0.000s
I20260809 14:30:24.339497 32705 tablet_bootstrap.cc:492] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:24.339915 32705 ts_tablet_manager.cc:1397] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.523s user 0.003s sys 0.011s
I20260809 14:30:24.355633 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff78e5cb7eb4796a8afa4847f0d2834" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:24.369252 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_44d99dd47a7a4e5ca147890da0688db7 [id=47adcc871eb749e4bb68dc8c8c9b9bc0]
W20260809 14:30:24.370325 32333 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47010: Aborted: Op aborted by new leader
I20260809 14:30:24.372889 32333 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47010 (request call id 329) took 2075 ms. Trace:
I20260809 14:30:24.378981 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_80c122d74d1d4ff598b1f8e97c766fcc [id=49782f8fd7b74990b01d83148e23af89]
I20260809 14:30:24.380805 32705 raft_consensus.cc:348] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.381696 32705 raft_consensus.cc:374] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:24.381963 32705 raft_consensus.cc:729] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:24.382606 32705 consensus_queue.cc:260] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.383620 1020 tablet_replica.cc:330] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:24.385455 1020 raft_consensus.cc:2229] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:24.306545 31933 catalog_manager.cc:4841] TS 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): tablet 94c4cbdda4ed48a8b34728d231497d02 (table default.Table_bb45fcc479b0477f872971659421b3ac [id=7043672a284145c7a5c9a5d80c322ef8]) successfully deleted
W20260809 14:30:24.389477 31933 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 82980.6us
E20260809 14:30:24.204408 31943 catalog_manager.cc:5650] Error updating tablets from 3eb0bc287b12421d89abf97ca66e4d9b: Service unavailable: leader is not yet ready. Tablet report was: is_incremental: true updated_tablets { tablet_id: "f74b4deeb32941aeaa3c4b813333c14e" state: RUNNING consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "32cb7842617a4779bf110e4ea254e011" state: RUNNING consensus_state { current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { } } updated_tablets { tablet_id: "239ad14cb3a642e08bace0759bc0b732" state: RUNNING consensus_state { current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26813 live_row_count: 0 } } updated_tablets { tablet_id: "2f171260cb764ac284aef85e4219767e" state: RUNNING consensus_state { current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26813 live_row_count: 0 } } updated_tablets { tablet_id: "43eeed4a20d6413f873585c95bb2858a" state: RUNNING consensus_state { current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26844 live_row_count: 0 } } updated_tablets { tablet_id: "94c4cbdda4ed48a8b34728d231497d02" state: RUNNING consensus_state { current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26844 live_row_count: 0 } } updated_tablets { tablet_id: "67afcb47ffc947c5b1d98c96de333116" state: RUNNING consensus_state { current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } } } schema_version: 2 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26844 live_row_count: 0 } } updated_tablets { tablet_id: "8b53f7d398d64701b4504db5c9c440b8" state: RUNNING consensus_state { current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: HEALTHY } } } } schema_version: 2 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 26844 live_row_count: 0 } } updated_tablets { tablet_id: "5121e9f5a85a45d285d78f41a442769a" state: RUNNING consensus_state { current_term: 1 leader_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } } } } schema_version: 1 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "88644048fa4a443fac09e951d78b3063" state: RUNNING consensus_state { current_term: 1 leader_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { } } removed_tablet_ids: "9922cad6d10945d5b013a168a3496676" sequence_number: 60
W20260809 14:30:24.391867 31943 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.30.252.66:37847 (request call id 61) took 1995 ms (client timeout 1000 ms). Trace:
W20260809 14:30:24.392216 31943 rpcz_store.cc:269] 0809 14:30:22.396884 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.408328 (+ 11444us) service_pool.cc:227] Handling call
0809 14:30:24.204198 (+1795870us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:24.204220 (+ 22us) write_op.cc:454] Released schema lock
0809 14:30:24.391751 (+187531us) inbound_call.cc:173] Queueing failure response
Related trace 'op':
0809 14:30:24.203956 (+ 0us) op_driver.cc:372] HandleFailure(Service unavailable: leader is not yet ready)
0809 14:30:24.203995 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0809 14:30:24.204035 (+ 40us) write_op.cc:454] Released schema lock
0809 14:30:24.204057 (+ 22us) write_op.cc:335] FINISH: Op aborted
Metrics: {"reported_tablets":10,"child_traces":[["op",{}]]}
W20260809 14:30:24.388298 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 120015us
I20260809 14:30:24.413727 1168 raft_consensus.cc:482] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bed14aff53a41b2bbf5a13ce55dec5a)
I20260809 14:30:24.432045 1020 raft_consensus.cc:2258] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:24.442051 996 ts_tablet_manager.cc:1918] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:24.442423 996 log.cc:1196] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/c468382a28934d50babf7bbc9a834c08
I20260809 14:30:24.443138 996 ts_tablet_manager.cc:1939] T c468382a28934d50babf7bbc9a834c08 P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:24.444294 1301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 14:30:24.449542 1214 raft_consensus.cc:482] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:24.450253 1214 raft_consensus.cc:504] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.454293 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3619d49d0b944a91bcb181e884047b38" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:24.455157 32443 leader_election.cc:341] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:24.455580 32443 leader_election.cc:302] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:24.456157 32439 leader_election.cc:341] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:24.457260 1214 leader_election.cc:288] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:24.471436 1264 raft_consensus.cc:2726] T 7856076c1332479a894d2d69d16b19f5 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:24.472534 32443 leader_election.cc:302] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:24.477699 1264 raft_consensus.cc:2726] T 5ff78e5cb7eb4796a8afa4847f0d2834 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:24.480105 1004 ts_tablet_manager.cc:1918] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 14:30:24.480542 1004 log.cc:1196] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/239ad14cb3a642e08bace0759bc0b732
I20260809 14:30:24.481395 1004 ts_tablet_manager.cc:1939] T 239ad14cb3a642e08bace0759bc0b732 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:24.483466 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 287) took 6086 ms. Trace:
I20260809 14:30:24.492956 1004 rpcz_store.cc:276] 0809 14:30:18.396653 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.398354 (+ 1701us) service_pool.cc:227] Handling call
0809 14:30:23.840049 (+5441695us) ts_tablet_manager.cc:1145] Deleting tablet 239ad14cb3a642e08bace0759bc0b732
0809 14:30:23.840095 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:23.967869 (+127774us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.473905 (+506036us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.483410 (+ 9505us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5405829}
I20260809 14:30:24.494881 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 291) took 5908 ms. Trace:
I20260809 14:30:24.495303 1004 rpcz_store.cc:276] 0809 14:30:18.586757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.586902 (+ 145us) service_pool.cc:227] Handling call
0809 14:30:24.494524 (+5907622us) ts_tablet_manager.cc:1145] Deleting tablet ea0a36a6ddc64557b31d0dfef8b5b457
0809 14:30:24.494568 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.494827 (+ 259us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5906944}
I20260809 14:30:24.496671 1004 tablet_replica.cc:330] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:24.497560 1004 raft_consensus.cc:2229] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:24.498101 1004 raft_consensus.cc:2258] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:24.499794 1004 ts_tablet_manager.cc:1905] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 14:30:24.509497 31933 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.252.125:42169 timed out after 0.552s (SENT) [suppressed 2 similar messages]
I20260809 14:30:24.416877 32330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88644048fa4a443fac09e951d78b3063" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:24.423127 32693 tablet_bootstrap.cc:492] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:24.526647 1004 ts_tablet_manager.cc:1918] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:24.527154 1004 log.cc:1196] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/f74b4deeb32941aeaa3c4b813333c14e
I20260809 14:30:24.535121 1004 ts_tablet_manager.cc:1939] T f74b4deeb32941aeaa3c4b813333c14e P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:24.537335 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 295) took 5619 ms. Trace:
I20260809 14:30:24.537812 1004 rpcz_store.cc:276] 0809 14:30:18.918291 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.918423 (+ 132us) service_pool.cc:227] Handling call
0809 14:30:24.496520 (+5578097us) ts_tablet_manager.cc:1145] Deleting tablet f74b4deeb32941aeaa3c4b813333c14e
0809 14:30:24.496562 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.508159 (+ 11597us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.526399 (+ 18240us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.537278 (+ 10879us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5517014}
I20260809 14:30:24.539755 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 304) took 5374 ms. Trace:
I20260809 14:30:24.540154 1004 rpcz_store.cc:276] 0809 14:30:19.165508 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:19.168291 (+ 2783us) service_pool.cc:227] Handling call
0809 14:30:24.539381 (+5371090us) ts_tablet_manager.cc:1145] Deleting tablet c4f4c1eb916c4b51a5e176d541da8574
0809 14:30:24.539429 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.539700 (+ 271us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5350885}
I20260809 14:30:24.541782 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 303) took 5376 ms. Trace:
I20260809 14:30:24.542171 1004 rpcz_store.cc:276] 0809 14:30:19.165228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:19.212747 (+ 47519us) service_pool.cc:227] Handling call
0809 14:30:24.541431 (+5328684us) ts_tablet_manager.cc:1145] Deleting tablet 04b74ba6259e47569d13dfe62d14a4af
0809 14:30:24.541485 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.541731 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5319539}
I20260809 14:30:24.543807 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 311) took 4464 ms. Trace:
I20260809 14:30:24.544193 1004 rpcz_store.cc:276] 0809 14:30:20.078990 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.080675 (+ 1685us) service_pool.cc:227] Handling call
0809 14:30:24.543437 (+4462762us) ts_tablet_manager.cc:1145] Deleting tablet 90e67f393f7a4bb987edc99aa86a9c90
0809 14:30:24.543490 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.543756 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4456486}
I20260809 14:30:24.545605 32691 tablet_bootstrap.cc:654] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:24.545819 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 310) took 4694 ms. Trace:
I20260809 14:30:24.480121 32705 ts_tablet_manager.cc:1428] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.140s user 0.003s sys 0.004s
I20260809 14:30:24.546974 32705 tablet_bootstrap.cc:492] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:24.546219 1004 rpcz_store.cc:276] 0809 14:30:19.851019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.077524 (+226505us) service_pool.cc:227] Handling call
0809 14:30:24.545488 (+4467964us) ts_tablet_manager.cc:1145] Deleting tablet 666bea53f14541368730f69402c2b411
0809 14:30:24.545528 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.545770 (+ 242us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4424584}
I20260809 14:30:24.554200 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 315) took 4097 ms. Trace:
I20260809 14:30:24.554606 1004 rpcz_store.cc:276] 0809 14:30:20.456791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.457051 (+ 260us) service_pool.cc:227] Handling call
0809 14:30:24.553840 (+4096789us) ts_tablet_manager.cc:1145] Deleting tablet b12d4849dec841a98e7f1a00c7d5f24e
0809 14:30:24.553885 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.554143 (+ 258us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4096173}
I20260809 14:30:24.556245 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 316) took 4062 ms. Trace:
I20260809 14:30:24.556671 1004 rpcz_store.cc:276] 0809 14:30:20.493574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.493924 (+ 350us) service_pool.cc:227] Handling call
0809 14:30:24.555831 (+4061907us) ts_tablet_manager.cc:1145] Deleting tablet 7ce3271ee8684662a9d3cd56157e572a
0809 14:30:24.555871 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.556192 (+ 321us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4061383}
I20260809 14:30:24.558365 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 317) took 4018 ms. Trace:
I20260809 14:30:24.558753 1004 rpcz_store.cc:276] 0809 14:30:20.540261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.540474 (+ 213us) service_pool.cc:227] Handling call
0809 14:30:24.557869 (+4017395us) ts_tablet_manager.cc:1145] Deleting tablet 200c48c11cb042169b307bc1902be82e
0809 14:30:24.557907 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.558315 (+ 408us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4016795}
I20260809 14:30:24.560482 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 320) took 3849 ms. Trace:
I20260809 14:30:24.560865 1004 rpcz_store.cc:276] 0809 14:30:20.711047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:20.711191 (+ 144us) service_pool.cc:227] Handling call
0809 14:30:24.559961 (+3848770us) ts_tablet_manager.cc:1145] Deleting tablet 2af4e169f518447c8d29c630dca93ff9
0809 14:30:24.560002 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.560405 (+ 403us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3847546}
I20260809 14:30:24.562572 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 330) took 3250 ms. Trace:
I20260809 14:30:24.563797 1004 rpcz_store.cc:276] 0809 14:30:21.312216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.312347 (+ 131us) service_pool.cc:227] Handling call
0809 14:30:24.562090 (+3249743us) ts_tablet_manager.cc:1145] Deleting tablet a0b196e429514cbc803e9a99a986d4d2
0809 14:30:24.562130 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.562521 (+ 391us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3249204}
*** SIGABRT (@0x3e800007bf1) received by PID 31729 (TID 0x7fa0f3efd700) from PID 31729; stack trace: ***
I20260809 14:30:24.572350 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 331) took 2999 ms. Trace:
I20260809 14:30:24.572729 1004 rpcz_store.cc:276] 0809 14:30:21.572969 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.579767 (+ 6798us) service_pool.cc:227] Handling call
0809 14:30:24.565077 (+2985310us) ts_tablet_manager.cc:1145] Deleting tablet c0fa3fb92dbf4ad6a5fa2b8472e376ab
0809 14:30:24.571986 (+ 6909us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.572292 (+ 306us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2983315}
@ 0x2b0af0 __tsan::CallUserSignalHandler() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b31e0 rtl_sigaction() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
I20260809 14:30:24.440850 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_ed7d64f6d7b644c48b0190ff65c56551 [id=4a72c13a29f74ba19c69e2392c122b72]
I20260809 14:30:24.578925 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_f3272d0c348d4d20bd9b5d958bf81581 [id=4ec3081646e84f14ba0605bbb51e2f11]
I20260809 14:30:24.581679 32693 ts_tablet_manager.cc:1397] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.724s user 0.006s sys 0.021s
I20260809 14:30:24.429050 32324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8b48c8cde4f11a53eec939a1f8e3d" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:24.432772 1168 raft_consensus.cc:504] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.584947 1168 leader_election.cc:288] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
W20260809 14:30:24.434763 32260 consensus_peers.cc:487] T e74db9caa7f94059b1bba4a082fe57ef P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Couldn't send request to peer 6bed14aff53a41b2bbf5a13ce55dec5a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e74db9caa7f94059b1bba4a082fe57ef. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:24.585726 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 151132us
I20260809 14:30:24.587309 32260 leader_election.cc:302] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:24.589576 1252 raft_consensus.cc:2726] T e28a765e1854486595683c0ac00da24c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:24.375886 32331 raft_consensus.cc:1261] T 8b53f7d398d64701b4504db5c9c440b8 P 3eb0bc287b12421d89abf97ca66e4d9b [term 2 FOLLOWER]: Refusing update from remote peer 6bed14aff53a41b2bbf5a13ce55dec5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 14:30:24.373697 32333 rpcz_store.cc:276] 0809 14:30:22.297428 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.297724 (+ 296us) service_pool.cc:227] Handling call
0809 14:30:24.372766 (+2075042us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:22.421250 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:22.421675 (+ 425us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:22.421705 (+ 30us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:22.421763 (+ 58us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:22.421874 (+ 111us) alter_schema_op.cc:126] START. Timestamp: 1786285822415337
0809 14:30:22.421891 (+ 17us) op_driver.cc:331] REPLICATION: starting
0809 14:30:22.641838 (+219947us) log.cc:842] Serialized 173 byte log entry
0809 14:30:23.146386 (+504548us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b1800139ae0. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8399f828b1 00007f8399f8f661 00007f839113c638 00007f839113fb11 00007f839113fac9 00007f839113fa59 00007f839113fa21 00007f839113ed1d 0000000000580784 00000000005805b9 00007f839111edc6
0809 14:30:23.156240 (+ 9854us) spinlock_profiling.cc:243] Waited 90 ms on lock 0x7b1800139ae0. stack: 00007f839110a033 00007f8391109f30 00007f8390c8b534 00000000004db2b4 00000000004db249 00007f8399f828b1 00007f8399f8f661 00007f83910b34ca 0000000000574009 00007f83974ddbcf 00007f839113c7b6 00007f839113fb11 00007f839113fac9 00007f839113fa59 00007f839113fa21 00007f839113ed1d
0809 14:30:24.348935 (+1192695us) op_driver.cc:444] REPLICATION: finished
0809 14:30:24.352784 (+ 3849us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 14:30:24.354382 (+ 1598us) alter_schema_op.cc:169] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"mutex_wait_us":102351,"prepare.queue_time_us":110740,"prepare.run_cpu_time_us":12406,"prepare.run_wall_time_us":470794,"raft.queue_time_us":89100,"raft.run_cpu_time_us":29,"raft.run_wall_time_us":41,"replication_time_us":1923299,"spinlock_wait_cycles":223910400,"thread_start_us":527655,"threads_started":2,"wal-append.queue_time_us":432768,"wal-append.run_cpu_time_us":1366,"wal-append.run_wall_time_us":1001399}]]}
I20260809 14:30:24.487587 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3619d49d0b944a91bcb181e884047b38" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:24.593927 32260 leader_election.cc:302] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b, 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:24.595095 1168 raft_consensus.cc:2781] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 14:30:24.595782 1168 raft_consensus.cc:686] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:24.597226 1168 consensus_queue.cc:237] T 0bad186f0ccf4c15a4f0b16bfffa945c P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.607235 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 332) took 2720 ms. Trace:
I20260809 14:30:24.607565 1004 rpcz_store.cc:276] 0809 14:30:21.886794 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.935122 (+ 48328us) service_pool.cc:227] Handling call
0809 14:30:24.573739 (+2638617us) ts_tablet_manager.cc:1145] Deleting tablet f43385579b29417289f3bdacf8fce278
0809 14:30:24.606712 (+ 32973us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.607165 (+ 453us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2616623}
I20260809 14:30:24.581075 1226 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47010 (request call id 298) took 3537 ms. Trace:
I20260809 14:30:24.570595 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 275) took 6399 ms. Trace:
I20260809 14:30:24.582142 32443 leader_election.cc:302] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:24.620383 32443 leader_election.cc:341] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:24.620785 32443 leader_election.cc:302] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:24.622133 1264 raft_consensus.cc:2726] T 88644048fa4a443fac09e951d78b3063 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:24.622819 1264 consensus_queue.cc:1035] T 8b53f7d398d64701b4504db5c9c440b8 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 14:30:24.624727 1214 raft_consensus.cc:2726] T 2af8b48c8cde4f11a53eec939a1f8e3d P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:24.625664 1214 consensus_queue.cc:1035] T a051520649c84898ae634f58f3fb42b3 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 14:30:24.639635 32705 tablet_bootstrap.cc:654] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:24.643991 820 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 8 FOLLOWER]: Refusing update from remote peer ee93a411a1b04d38998018f8c6be59e0: Log matching property violated. Preceding OpId in replica: term: 6 index: 320. Preceding OpId from leader: term: 8 index: 477. (index mismatch)
I20260809 14:30:24.645845 1294 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 477, Last known committed idx: 320, Time since last communication: 0.001s
I20260809 14:30:24.644614 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_06a4ae5a60b44618a6f88f5ad5b71ac9 [id=4ecde42868384f93a2abe6720fdc3966]
I20260809 14:30:24.662367 32693 raft_consensus.cc:348] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.663059 32693 raft_consensus.cc:374] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:24.663290 32693 raft_consensus.cc:729] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
I20260809 14:30:24.663844 32693 consensus_queue.cc:260] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.612653 1226 rpcz_store.cc:276] 0809 14:30:21.043380 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.043556 (+ 176us) service_pool.cc:227] Handling call
0809 14:30:24.580976 (+3537420us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:21.109020 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:21.109576 (+ 556us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:21.109690 (+ 114us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:21.109963 (+ 273us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:21.110106 (+ 143us) alter_schema_op.cc:126] START. Timestamp: 1786285821110010
0809 14:30:21.110181 (+ 75us) op_driver.cc:331] REPLICATION: starting
0809 14:30:21.191575 (+ 81394us) log.cc:842] Serialized 173 byte log entry
0809 14:30:24.005748 (+2814173us) op_driver.cc:444] REPLICATION: finished
0809 14:30:24.010219 (+ 4471us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:24.011423 (+ 1204us) log.cc:842] Serialized 5 byte log entry
0809 14:30:24.516592 (+505169us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.517956 (+ 1364us) log.cc:842] Serialized 14 byte log entry
0809 14:30:24.580758 (+ 62802us) alter_schema_op.cc:71] Released schema lock
0809 14:30:24.580785 (+ 27us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4296,"prepare.queue_time_us":63056,"prepare.run_cpu_time_us":22394,"prepare.run_wall_time_us":1319737,"replication_time_us":2895450,"thread_start_us":1394373,"threads_started":3,"wal-append.queue_time_us":978371,"wal-append.run_cpu_time_us":1496,"wal-append.run_wall_time_us":1020356}]]}
I20260809 14:30:24.672343 1004 tablet_replica.cc:330] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:24.673199 1004 raft_consensus.cc:2229] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:24.673740 1004 raft_consensus.cc:2258] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:24.675690 1004 ts_tablet_manager.cc:1905] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:24.661965 1285 tablet.cc:1722] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:24.632812 996 rpcz_store.cc:276] 0809 14:30:18.171222 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.174097 (+ 2875us) service_pool.cc:227] Handling call
0809 14:30:23.675979 (+5501882us) ts_tablet_manager.cc:1145] Deleting tablet c468382a28934d50babf7bbc9a834c08
0809 14:30:23.687852 (+ 11873us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.061030 (+373178us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.441664 (+380634us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.570522 (+128858us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":124984,"tablet-delete.queue_time_us":5460380}
I20260809 14:30:24.688405 32693 ts_tablet_manager.cc:1428] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.106s user 0.004s sys 0.003s
I20260809 14:30:24.694979 996 tablet_replica.cc:330] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
W20260809 14:30:24.698545 32443 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.66:40891, user_credentials={real_user=slave}} blocked reactor thread for 76779.8us
W20260809 14:30:24.699283 32443 leader_election.cc:341] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:24.699708 32443 leader_election.cc:302] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:24.700549 1214 raft_consensus.cc:2726] T 3619d49d0b944a91bcb181e884047b38 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:24.689440 32693 tablet_bootstrap.cc:492] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
@ 0x7fa108c30980 (unknown) at ??:0
I20260809 14:30:24.727617 1233 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47010 (request call id 310) took 3277 ms. Trace:
I20260809 14:30:24.733244 996 raft_consensus.cc:2229] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:24.733760 996 raft_consensus.cc:2258] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:24.735759 996 ts_tablet_manager.cc:1905] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:24.741232 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_eaa055a1bfde40768487fe94df03d134 [id=4ff5e70bfad84d4b88d5966188c69279]
I20260809 14:30:24.744019 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_4878b3fdb4ad4a479d83e997033a54ed [id=503a9c2526e74ac690913ac0c64cf4f9]
I20260809 14:30:24.769214 32691 tablet_bootstrap.cc:492] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:24.769726 32691 ts_tablet_manager.cc:1397] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.616s user 0.011s sys 0.005s
I20260809 14:30:24.774917 32691 raft_consensus.cc:348] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.775548 32691 raft_consensus.cc:374] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:24.775784 32691 raft_consensus.cc:729] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:24.792789 1020 ts_tablet_manager.cc:1905] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:24.809368 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_853333f626c34bc18f5a1245314d7e99 [id=504e8c38c3f1441ebe085ccbcbdab567]
@ 0x7fa103eede87 gsignal at ??:0
I20260809 14:30:24.827598 32693 tablet_bootstrap.cc:654] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:24.851428 32705 tablet_bootstrap.cc:492] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:24.851897 32705 ts_tablet_manager.cc:1397] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.305s user 0.010s sys 0.005s
I20260809 14:30:24.855295 32705 raft_consensus.cc:348] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.855999 32705 raft_consensus.cc:374] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:24.728628 1233 rpcz_store.cc:276] 0809 14:30:21.450383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:21.450516 (+ 133us) service_pool.cc:227] Handling call
0809 14:30:24.727520 (+3277004us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 14:30:21.451535 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0809 14:30:21.451892 (+ 357us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0809 14:30:21.451926 (+ 34us) alter_schema_op.cc:65] Acquired schema lock
0809 14:30:21.451994 (+ 68us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0809 14:30:21.452113 (+ 119us) alter_schema_op.cc:126] START. Timestamp: 1786285821452009
0809 14:30:21.452132 (+ 19us) op_driver.cc:331] REPLICATION: starting
0809 14:30:21.452722 (+ 590us) log.cc:842] Serialized 173 byte log entry
0809 14:30:24.028088 (+2575366us) op_driver.cc:444] REPLICATION: finished
0809 14:30:24.051085 (+ 22997us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0809 14:30:24.322498 (+271413us) log.cc:842] Serialized 5 byte log entry
0809 14:30:24.656426 (+333928us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:24.715790 (+ 59364us) log.cc:842] Serialized 14 byte log entry
0809 14:30:24.718246 (+ 2456us) alter_schema_op.cc:71] Released schema lock
0809 14:30:24.727325 (+ 9079us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":22817,"prepare.queue_time_us":124,"prepare.run_cpu_time_us":34976,"prepare.run_wall_time_us":969117,"replication_time_us":2575879,"thread_start_us":1076263,"threads_started":2,"wal-append.queue_time_us":886737,"wal-append.run_cpu_time_us":2966,"wal-append.run_wall_time_us":1101631}]]}
I20260809 14:30:24.856233 32705 raft_consensus.cc:729] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:24.873377 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_eb4a12a97b084ef2963e4341cb88fb7a [id=51db24bf22c4442398d89b168f238d02]
W20260809 14:30:24.877434 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 273012us
@ 0x7fa103eef7f1 abort at ??:0
@ 0x2b4f37 __interceptor_abort at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
I20260809 14:30:24.882942 32691 consensus_queue.cc:260] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.892581 1308 raft_consensus.cc:482] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:24.893108 1308 raft_consensus.cc:504] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.896113 32705 consensus_queue.cc:260] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:24.896708 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4980140ba2a4ecc87001e2a43c8e813" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:24.898131 32705 ts_tablet_manager.cc:1428] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.046s user 0.004s sys 0.002s
I20260809 14:30:24.898172 1319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:24.898640 1319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10512ac8b3406098fad1f7da012ab3" member_type: VOTER last_known_addr { host: "127.30.252.126" port: 43223 } } peers { permanent_uuid: "852dbd0962ae4211923851c7a632768e" member_type: VOTER last_known_addr { host: "127.30.252.125" port: 42169 } } peers { permanent_uuid: "ee93a411a1b04d38998018f8c6be59e0" member_type: VOTER last_known_addr { host: "127.30.252.124" port: 34903 } }
W20260809 14:30:24.899829 32439 leader_election.cc:341] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:24.901415 820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "852dbd0962ae4211923851c7a632768e" candidate_term: 9 candidate_status { last_received { term: 7 index: 476 } } ignore_live_leader: false dest_uuid: "9d10512ac8b3406098fad1f7da012ab3" is_pre_election: true
I20260809 14:30:24.902191 1308 leader_election.cc:288] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
I20260809 14:30:24.903501 1319 leader_election.cc:288] T 00000000000000000000000000000000 P 852dbd0962ae4211923851c7a632768e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9d10512ac8b3406098fad1f7da012ab3 (127.30.252.126:43223), ee93a411a1b04d38998018f8c6be59e0 (127.30.252.124:34903)
I20260809 14:30:24.908906 32705 tablet_bootstrap.cc:492] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:24.943610 32705 tablet_bootstrap.cc:654] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:24.949438 32169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273ca6176a184a719538a9455afded4d" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:24.951748 32333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4980140ba2a4ecc87001e2a43c8e813" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
W20260809 14:30:24.954826 32443 leader_election.cc:341] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:24.955247 32443 leader_election.cc:302] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:24.957006 32507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6ebda27b44048b5734e8798a980c2" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" is_pre_election: true
I20260809 14:30:24.960214 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6ebda27b44048b5734e8798a980c2" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:24.960472 32165 raft_consensus.cc:3037] T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:24.965626 1308 raft_consensus.cc:2726] T d4980140ba2a4ecc87001e2a43c8e813 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:24.975714 1302 consensus_queue.cc:1046] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260809 14:30:24.980970 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_11709a3dc2554d2f98756c54dba95747 [id=5243da85a5074651adbde8e65a0a16e2]
I20260809 14:30:24.978070 1311 consensus_queue.cc:1046] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260809 14:30:25.000984 32691 ts_tablet_manager.cc:1428] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.231s user 0.006s sys 0.002s
I20260809 14:30:25.003021 1168 raft_consensus.cc:3032] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Stepping down as leader of term 1
I20260809 14:30:25.003338 1168 raft_consensus.cc:729] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:25.013636 1316 raft_consensus.cc:3032] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Stepping down as leader of term 1
I20260809 14:30:25.014034 1316 raft_consensus.cc:729] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Running, Role: LEADER
I20260809 14:30:25.014799 1316 consensus_queue.cc:260] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.017091 1316 raft_consensus.cc:3037] T b6f17af7f81741c68bafa73c2c3d91af P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:25.021505 32691 tablet_bootstrap.cc:492] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:25.036079 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_7f6aaf330b564e2c8a3126bca9401698 [id=567557fc695846c0aa78b86becb353bc]
I20260809 14:30:25.037503 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_1a59c65c45d844c6ad1424a0c36de6e7 [id=57abcf2570f14f039173f115a89c2d4d]
I20260809 14:30:25.004094 1168 consensus_queue.cc:260] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.051201 1168 raft_consensus.cc:3037] T 43eeed4a20d6413f873585c95bb2858a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Advancing to term 2
I20260809 14:30:25.055903 1308 raft_consensus.cc:482] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:25.056674 1308 raft_consensus.cc:504] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.061272 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5634154c6f149d3b2d36b961aee38e0" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
W20260809 14:30:25.063292 32439 leader_election.cc:341] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:25.063119 32324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5634154c6f149d3b2d36b961aee38e0" candidate_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" is_pre_election: true
I20260809 14:30:25.064128 1308 leader_election.cc:288] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891)
W20260809 14:30:25.065111 32443 leader_election.cc:341] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3eb0bc287b12421d89abf97ca66e4d9b (127.30.252.66:40891): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:25.065506 32443 leader_election.cc:302] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bed14aff53a41b2bbf5a13ce55dec5a; no voters: 3eb0bc287b12421d89abf97ca66e4d9b, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:25.075635 31933 leader_election.cc:334] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169): Timed out: RequestConsensusVote RPC to 127.30.252.125:42169 timed out after 0.552s (SENT)
W20260809 14:30:25.082101 1321 log.cc:924] Time spent T 32cb7842617a4779bf110e4ea254e011 P e1f94e0d792c46c9a0b6ab42820da6eb: Append to log took a long time: real 0.062s user 0.000s sys 0.001s
I20260809 14:30:25.093290 1308 raft_consensus.cc:2726] T a5634154c6f149d3b2d36b961aee38e0 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:25.101382 819 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 9d10512ac8b3406098fad1f7da012ab3 [term 8 FOLLOWER]: Rejecting Update request from peer 852dbd0962ae4211923851c7a632768e for earlier term 7. Current term is 8. Ops: [6.321-7.476]
I20260809 14:30:25.025786 1320 tablet.cc:1722] T 239ad14cb3a642e08bace0759bc0b732 P 6bed14aff53a41b2bbf5a13ce55dec5a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260809 14:30:25.115868 32693 tablet_bootstrap.cc:492] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:25.116271 32693 ts_tablet_manager.cc:1397] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.427s user 0.006s sys 0.015s
I20260809 14:30:25.117138 32260 leader_election.cc:302] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:25.132730 32260 leader_election.cc:302] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
W20260809 14:30:25.133880 32260 consensus_peers.cc:487] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dc8d0cad33b4778af9fe9247743c38f. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:25.140379 1302 raft_consensus.cc:2726] T ae1af25739774df8a5536c75c2e1f416 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:30:25.142982 32260 consensus_peers.cc:487] T 239ad14cb3a642e08bace0759bc0b732 P 3eb0bc287b12421d89abf97ca66e4d9b -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 14:30:25.144541 1261 raft_consensus.cc:2726] T 5121e9f5a85a45d285d78f41a442769a P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 14:30:25.148342 31933 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 23d2b217278c481e8464d00188c70048 (table default.Table_ed7d64f6d7b644c48b0190ff65c56551 [id=4a72c13a29f74ba19c69e2392c122b72], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 23d2b217278c481e8464d00188c70048 consensus state has no leader
@ 0x7fa106f21df4 google::LogMessage::SendToLog() at ??:0
I20260809 14:30:25.180869 1004 ts_tablet_manager.cc:1918] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:25.181277 1004 log.cc:1196] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/94c4cbdda4ed48a8b34728d231497d02
I20260809 14:30:25.181910 1004 ts_tablet_manager.cc:1939] T 94c4cbdda4ed48a8b34728d231497d02 P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:25.197567 32693 raft_consensus.cc:348] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.198232 32693 raft_consensus.cc:374] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:25.198465 32693 raft_consensus.cc:729] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0bc287b12421d89abf97ca66e4d9b, State: Initialized, Role: FOLLOWER
W20260809 14:30:25.206936 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 63518.3us
W20260809 14:30:25.207705 32260 leader_election.cc:341] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Illegal state: must be running to vote when last-logged opid is not known
I20260809 14:30:25.208511 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_4eccea50c485475c9e4a6f9e9990cac1 [id=5aef0a1444ad4cba9e08edb834962018]
W20260809 14:30:25.210333 32439 consensus_peers.cc:487] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a -> Peer e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831): Couldn't send request to peer e1f94e0d792c46c9a0b6ab42820da6eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 14:30:25.210700 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 72351.2us
I20260809 14:30:25.215332 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 333) took 3146 ms. Trace:
I20260809 14:30:25.215667 1004 rpcz_store.cc:276] 0809 14:30:22.068769 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.111790 (+ 43021us) service_pool.cc:227] Handling call
0809 14:30:24.608441 (+2496651us) ts_tablet_manager.cc:1145] Deleting tablet 94c4cbdda4ed48a8b34728d231497d02
0809 14:30:24.672204 (+ 63763us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:24.813479 (+141275us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:25.116246 (+302767us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:25.215254 (+ 99008us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2495997}
I20260809 14:30:25.238030 1264 consensus_queue.cc:1035] T 5ac7afd531a746cab7618ead98b4c4d9 P 6bed14aff53a41b2bbf5a13ce55dec5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 14:30:25.239279 1326 log.cc:924] Time spent T 32cb7842617a4779bf110e4ea254e011 P 6bed14aff53a41b2bbf5a13ce55dec5a: Append to log took a long time: real 0.152s user 0.001s sys 0.000s
I20260809 14:30:25.243880 32691 tablet_bootstrap.cc:654] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:25.234760 32705 tablet_bootstrap.cc:492] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:25.253147 32705 ts_tablet_manager.cc:1397] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.344s user 0.009s sys 0.003s
@ 0x7fa106f229a0 google::LogMessage::Flush() at ??:0
I20260809 14:30:25.199031 32693 consensus_queue.cc:260] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:25.272881 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 53013.8us
I20260809 14:30:25.274257 996 ts_tablet_manager.cc:1918] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 14:30:25.274703 996 log.cc:1196] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-2/wal/wals/0bad186f0ccf4c15a4f0b16bfffa945c
I20260809 14:30:25.275354 996 ts_tablet_manager.cc:1939] T 0bad186f0ccf4c15a4f0b16bfffa945c P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting consensus metadata
I20260809 14:30:25.277570 819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39036 (request call id 3) took 1167 ms. Trace:
I20260809 14:30:25.277912 819 rpcz_store.cc:276] 0809 14:30:24.110385 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:24.111272 (+ 887us) service_pool.cc:227] Handling call
0809 14:30:24.690944 (+579672us) raft_consensus.cc:1457] Updating replica for 156 ops
0809 14:30:25.102110 (+411166us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 14:30:25.277483 (+175373us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 14:30:25.281651 1004 tablet_replica.cc:330] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: stopping tablet replica
I20260809 14:30:25.282625 1004 raft_consensus.cc:2229] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:25.283118 1004 raft_consensus.cc:2258] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 14:30:25.294307 32693 ts_tablet_manager.cc:1428] T faf222c49c934d52a2642f20e8575702 P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent starting tablet: real 0.163s user 0.002s sys 0.004s
I20260809 14:30:25.294379 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_6cabfb77009541cabeb67c8fd017826e [id=5d43af6fc9cd412d873da5c46ee7db30]
I20260809 14:30:25.295301 32693 tablet_bootstrap.cc:492] T 94ff6ff27f6144d08d68a5e79c12963f P 3eb0bc287b12421d89abf97ca66e4d9b: Bootstrap starting.
I20260809 14:30:25.305224 1004 ts_tablet_manager.cc:1905] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:25.321058 1020 ts_tablet_manager.cc:1918] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 14:30:25.321472 1020 log.cc:1196] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-1/wal/wals/e846e795144e48e3b2a1eca8c1b26489
I20260809 14:30:25.322263 1020 ts_tablet_manager.cc:1939] T e846e795144e48e3b2a1eca8c1b26489 P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting consensus metadata
I20260809 14:30:25.328773 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_f13ec0ce721048f4aabef02eb1f5cc44 [id=617231ca8be54c389a6c0d821b081eb2]
I20260809 14:30:25.330364 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_04910b92ceac48bc8e23618fa93fc47a [id=6215d77a5b56410e995c95c88cecfed3]
I20260809 14:30:25.331636 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_f2894c35d51a436fb8ff7bc574624581 [id=65379b49137945acb43d46f22b12a128]
I20260809 14:30:25.332978 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_de805f1551304f6a92cf7d4db934dcda [id=653ab7bb85544b38a370d0a7b61a9714]
I20260809 14:30:25.335708 996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44626 (request call id 272) took 7175 ms. Trace:
I20260809 14:30:25.336017 996 rpcz_store.cc:276] 0809 14:30:18.160351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:18.213931 (+ 53580us) service_pool.cc:227] Handling call
0809 14:30:24.694798 (+6480867us) ts_tablet_manager.cc:1145] Deleting tablet 0bad186f0ccf4c15a4f0b16bfffa945c
0809 14:30:24.694870 (+ 72us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:25.073959 (+379089us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:25.274018 (+200059us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:25.335648 (+ 61630us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":37265,"tablet-delete.queue_time_us":6480229}
I20260809 14:30:25.348596 32691 tablet_bootstrap.cc:492] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:25.349134 32691 ts_tablet_manager.cc:1397] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent bootstrapping tablet: real 0.328s user 0.015s sys 0.000s
@ 0x7fa106f27adb google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260809 14:30:25.377630 32705 raft_consensus.cc:348] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.378423 32705 raft_consensus.cc:374] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:25.378679 32705 raft_consensus.cc:729] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:25.381356 32165 raft_consensus.cc:3037] T e74db9caa7f94059b1bba4a082fe57ef P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:25.385984 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_ac82ee262e4a40bcb9bc60cc74a8861f [id=65aaec21b1e5496a9b094e61bbd4f46a]
@ 0x357eb6 kudu::MasterStressTest::ReplaceTabletThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x3874ce _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3840
I20260809 14:30:25.379554 996 tablet_replica.cc:330] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a: stopping tablet replica
I20260809 14:30:25.398314 996 raft_consensus.cc:2229] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 14:30:25.398942 996 raft_consensus.cc:2258] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x387381 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:281
W20260809 14:30:25.407660 32439 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.65:41831, user_credentials={real_user=slave}} blocked reactor thread for 58447.5us
@ 0x386acb std::__1::__thread_proxy<>() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b11b0 __tsan_thread_start_func at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
I20260809 14:30:25.403120 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_348f7a78a51f40369f9fbd23580ff6b1 [id=667197f2e47b4910b1d8b1d3dd7f955d]
I20260809 14:30:25.416442 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_569de184b14243abb58d1414941be5aa [id=67767d6db83f4ae6ab265b854d2ad695]
I20260809 14:30:25.418973 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_153b78d9a7934c7ca382c10459415af0 [id=6a339b319fcc44de9e1a5feae512ba05]
I20260809 14:30:25.431037 1020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 248) took 9343 ms. Trace:
I20260809 14:30:25.420356 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_f4339baad1a14705a1f3f35db5081a8b [id=6c55b37d949a4648bd2d31852bb7727d]
@ 0x7fa108c256db start_thread at ??:0
@ 0x7fa103fd061f clone at ??:0
I20260809 14:30:25.463876 1004 ts_tablet_manager.cc:1918] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260809 14:30:25.519915 31933 leader_election.cc:334] T 00000000000000000000000000000000 P ee93a411a1b04d38998018f8c6be59e0 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 852dbd0962ae4211923851c7a632768e (127.30.252.125:42169): Timed out: RequestConsensusVote RPC to 127.30.252.125:42169 timed out after 0.702s (SENT)
I20260809 14:30:25.624500 1004 log.cc:1196] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting WAL directory at /tmp/dist-test-taskyetPbO/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786285777174086-31729-0/minicluster-data/ts-0/wal/wals/3d592b1c4520467791ffd9d36e0d97fd
W20260809 14:30:25.624966 31933 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.125:42169, user_credentials={real_user=slave}} blocked reactor thread for 105085us
I20260809 14:30:25.625365 1004 ts_tablet_manager.cc:1939] T 3d592b1c4520467791ffd9d36e0d97fd P e1f94e0d792c46c9a0b6ab42820da6eb: Deleting consensus metadata
I20260809 14:30:25.627401 1004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41544 (request call id 334) took 2807 ms. Trace:
I20260809 14:30:25.627717 1004 rpcz_store.cc:276] 0809 14:30:22.820358 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:22.820529 (+ 171us) service_pool.cc:227] Handling call
0809 14:30:25.281443 (+2460914us) ts_tablet_manager.cc:1145] Deleting tablet 3d592b1c4520467791ffd9d36e0d97fd
0809 14:30:25.281502 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:25.433868 (+152366us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:25.463654 (+ 29786us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:25.627338 (+163684us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2460297}
I20260809 14:30:25.516211 32705 consensus_queue.cc:260] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.630030 32705 ts_tablet_manager.cc:1428] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.377s user 0.006s sys 0.000s
I20260809 14:30:25.630877 32705 tablet_bootstrap.cc:492] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:25.774718 32705 tablet_bootstrap.cc:654] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:25.447907 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_65d17735090446738a487e765aef028a [id=6d697fa0067f4a3c960023c24567477c]
I20260809 14:30:25.431468 1020 rpcz_store.cc:276] 0809 14:30:16.087861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:16.090825 (+ 2964us) service_pool.cc:227] Handling call
0809 14:30:24.354713 (+8263888us) ts_tablet_manager.cc:1145] Deleting tablet e846e795144e48e3b2a1eca8c1b26489
0809 14:30:24.376919 (+ 22206us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:25.068172 (+691253us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:25.320814 (+252642us) tablet_metadata.cc:641] Metadata flushed
0809 14:30:25.430941 (+110127us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":45532,"tablet-delete.queue_time_us":8261468}
I20260809 14:30:25.782261 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_b342ca5d31c14a89baa7ed0ea2bf8da8 [id=6efa71f406734c998668150f80e73f12]
I20260809 14:30:25.783254 1020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 250) took 8557 ms. Trace:
I20260809 14:30:25.783715 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_222d7926360d490abc0717b23e7ff1f5 [id=6f6a067cb2a54359a9bfd2267b554581]
I20260809 14:30:25.783578 1020 rpcz_store.cc:276] 0809 14:30:17.226181 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.226698 (+ 517us) service_pool.cc:227] Handling call
0809 14:30:25.782794 (+8556096us) ts_tablet_manager.cc:1145] Deleting tablet 94ff6ff27f6144d08d68a5e79c12963f
0809 14:30:25.782856 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:25.783195 (+ 339us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8555532}
I20260809 14:30:25.784760 1020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47010 (request call id 263) took 8357 ms. Trace:
I20260809 14:30:25.785081 1020 rpcz_store.cc:276] 0809 14:30:17.427578 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 14:30:17.428086 (+ 508us) service_pool.cc:227] Handling call
0809 14:30:25.784377 (+8356291us) ts_tablet_manager.cc:1145] Deleting tablet 9e38a13e6e1c480880eab4bc2d2113cd
0809 14:30:25.784428 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 14:30:25.784712 (+ 284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8354546}
I20260809 14:30:25.786000 1020 tablet_replica.cc:330] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: stopping tablet replica
I20260809 14:30:25.786753 1020 raft_consensus.cc:2229] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 14:30:25.787519 1020 raft_consensus.cc:2258] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 14:30:25.513451 32260 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.30.252.126:43223, user_credentials={real_user=slave}} blocked reactor thread for 69581.7us
W20260809 14:30:25.788796 32260 leader_election.cc:341] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797): Not found: Tablet not found: 273ca6176a184a719538a9455afded4d
I20260809 14:30:25.789151 32260 leader_election.cc:302] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:25.479270 996 ts_tablet_manager.cc:1905] T 2dc8d0cad33b4778af9fe9247743c38f P 6bed14aff53a41b2bbf5a13ce55dec5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:25.790164 32260 leader_election.cc:302] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3eb0bc287b12421d89abf97ca66e4d9b; no voters: 6bed14aff53a41b2bbf5a13ce55dec5a, e1f94e0d792c46c9a0b6ab42820da6eb
I20260809 14:30:25.584863 32693 tablet_bootstrap.cc:654] T 94ff6ff27f6144d08d68a5e79c12963f P 3eb0bc287b12421d89abf97ca66e4d9b: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:25.589203 1168 raft_consensus.cc:482] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 14:30:25.791474 1168 raft_consensus.cc:504] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
W20260809 14:30:25.606817 31842 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.252.124:34903 timed out after 0.703s (SENT)
I20260809 14:30:25.793732 1316 raft_consensus.cc:2726] T 273ca6176a184a719538a9455afded4d P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 14:30:25.794232 1316 raft_consensus.cc:2726] T ebc6ebda27b44048b5734e8798a980c2 P 3eb0bc287b12421d89abf97ca66e4d9b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 14:30:25.794306 1168 leader_election.cc:288] T 4cdfc56edcd34696a7b806b0ee34d594 P 3eb0bc287b12421d89abf97ca66e4d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f94e0d792c46c9a0b6ab42820da6eb (127.30.252.65:41831), 6bed14aff53a41b2bbf5a13ce55dec5a (127.30.252.67:44797)
I20260809 14:30:25.498343 32691 raft_consensus.cc:348] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.795202 32691 raft_consensus.cc:374] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:25.795456 32691 raft_consensus.cc:729] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bed14aff53a41b2bbf5a13ce55dec5a, State: Initialized, Role: FOLLOWER
I20260809 14:30:25.796036 32691 consensus_queue.cc:260] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.797824 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_9f890199a5164be68c5b41ad6bc0ddb0 [id=6f6f1690ef7d4210bf4ba2aed598a88f]
I20260809 14:30:25.797914 32691 ts_tablet_manager.cc:1428] T eba8c216630d482db5bf235b4c8f5b15 P 6bed14aff53a41b2bbf5a13ce55dec5a: Time spent starting tablet: real 0.448s user 0.007s sys 0.003s
I20260809 14:30:25.798768 32691 tablet_bootstrap.cc:492] T 03f6a524d4ef48c4a31e0bfec2f557bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Bootstrap starting.
I20260809 14:30:25.799949 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_bb45fcc479b0477f872971659421b3ac [id=7043672a284145c7a5c9a5d80c322ef8]
W20260809 14:30:25.805126 31842 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.252.124:34903, user_credentials={real_user=slave}} blocked reactor thread for 198428us
I20260809 14:30:25.811146 1020 ts_tablet_manager.cc:1905] T 2dc8d0cad33b4778af9fe9247743c38f P 3eb0bc287b12421d89abf97ca66e4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 14:30:25.817375 32691 tablet_bootstrap.cc:654] T 03f6a524d4ef48c4a31e0bfec2f557bb P 6bed14aff53a41b2bbf5a13ce55dec5a: Neither blocks nor log segments found. Creating new log.
W20260809 14:30:25.838393 32260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.252.67:44797, user_credentials={real_user=slave}} blocked reactor thread for 47352.7us
I20260809 14:30:25.853453 32693 tablet_bootstrap.cc:492] T 94ff6ff27f6144d08d68a5e79c12963f P 3eb0bc287b12421d89abf97ca66e4d9b: No bootstrap required, opened a new log
I20260809 14:30:25.853812 32693 ts_tablet_manager.cc:1397] T 94ff6ff27f6144d08d68a5e79c12963f P 3eb0bc287b12421d89abf97ca66e4d9b: Time spent bootstrapping tablet: real 0.559s user 0.014s sys 0.004s
I20260809 14:30:25.855799 32693 raft_consensus.cc:348] T 94ff6ff27f6144d08d68a5e79c12963f P 3eb0bc287b12421d89abf97ca66e4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.865720 32165 raft_consensus.cc:3037] T 88644048fa4a443fac09e951d78b3063 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 14:30:25.867460 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdfc56edcd34696a7b806b0ee34d594" candidate_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" is_pre_election: true
I20260809 14:30:25.870301 32691 tablet_bootstrap.cc:492] T 03f6a524d4ef48c4a31e0bfec2f557bb P 6bed14aff53a41b2bbf5a13ce55dec5a: No bootstrap required, opened a new log
I20260809 14:30:25.887192 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_43f73cfcef6546fb985efbe413dff035 [id=72bef9dd3c5e43dc9e77990689fb7732]
I20260809 14:30:25.888334 32705 tablet_bootstrap.cc:492] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:25.914952 32705 ts_tablet_manager.cc:1397] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.284s user 0.016s sys 0.003s
I20260809 14:30:25.915922 1242 catalog_manager.cc:669] Loaded metadata for table default.Table_d49b8d515bb247e2b2114653b0d60633 [id=76f8e7a7e1114dd390d2a206ce0868f5]
I20260809 14:30:25.917868 32705 raft_consensus.cc:348] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.918936 32705 raft_consensus.cc:374] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 14:30:25.919431 32705 raft_consensus.cc:729] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f94e0d792c46c9a0b6ab42820da6eb, State: Initialized, Role: FOLLOWER
I20260809 14:30:25.920011 32705 consensus_queue.cc:260] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }
I20260809 14:30:25.921950 32705 ts_tablet_manager.cc:1428] T 9922cad6d10945d5b013a168a3496676 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 14:30:25.922744 32705 tablet_bootstrap.cc:492] T 5ac7afd531a746cab7618ead98b4c4d9 P e1f94e0d792c46c9a0b6ab42820da6eb: Bootstrap starting.
I20260809 14:30:25.933672 32705 tablet_bootstrap.cc:654] T 5ac7afd531a746cab7618ead98b4c4d9 P e1f94e0d792c46c9a0b6ab42820da6eb: Neither blocks nor log segments found. Creating new log.
I20260809 14:30:25.950354 32705 tablet_bootstrap.cc:492] T 5ac7afd531a746cab7618ead98b4c4d9 P e1f94e0d792c46c9a0b6ab42820da6eb: No bootstrap required, opened a new log
I20260809 14:30:25.950776 32705 ts_tablet_manager.cc:1397] T 5ac7afd531a746cab7618ead98b4c4d9 P e1f94e0d792c46c9a0b6ab42820da6eb: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.001s
I20260809 14:30:25.952883 32705 raft_consensus.cc:348] T 5ac7afd531a746cab7618ead98b4c4d9 P e1f94e0d792c46c9a0b6ab42820da6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1f94e0d792c46c9a0b6ab42820da6eb" member_type: VOTER last_known_addr { host: "127.30.252.65" port: 41831 } } peers { permanent_uuid: "3eb0bc287b12421d89abf97ca66e4d9b" member_type: VOTER last_known_addr { host: "127.30.252.66" port: 40891 } } peers { permanent_uuid: "6bed14aff53a41b2bbf5a13ce55dec5a" member_type: VOTER last_known_addr { host: "127.30.252.67" port: 44797 } }